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ADDRESSING  POINT  OF  NEED  IN  INTERACTIVE  MULTIMEDIA 
INSTRUCTION:  A  CONCEPTUAL  REVIEW  AND  EVALUATION 

EXECUTIVE  SUMMARY 


Research  Requirement: 

The  Maneuver  Center  of  Excellence  (MCoE)  Directorate  of  Training  and  Doctrine 
(DOTD),  Fort  Benning,  GA,  requested  research  to  address  how  the  Army  Learning  Model’s 
point  of  need  (PON)  concept  could  be  applied  in  the  design  and  development  of  interactive 
multimedia  instruction  (IMI)  exemplars.  A  first  step  in  that  process  was  to  review  and  evaluate 
existing  Army  IMI  which  is  the  focus  of  this  report.  Point  of  need  refers  both  (a)  to  the 
accessibility  of  learning  resources  at  any  time  or  place  and  (b)  to  the  ability  of  learning  resources 
to  provide  instruction  tailored  to  individual  learners’  level  of  knowledge  and  skill  as  well  as  their 
present  professional  requirements.  This  research  focused  on  the  latter  challenge;  namely,  we 
reviewed  existing  Army  IMI — one  type  of  digital  learning  resource — and  the  current  tailored 
training  literature  to  identify  design  features  and  principles  that  could  be  incorporated  into  Army 
IMI  and  would  enhance  its  ability  to  address  the  learning  requirements  of  a  specific  audience. 
While  we  focused  on  IMI  many  of  the  principles  and  techniques  we  identify  may  be  applicable 
to  other  types  of  digital  learning  resources. 

Procedure: 

To  determine  how  to  apply  Army  Learning  Model  (ALM)  point  of  need  and  tailored 
training  principles  in  Army  IMI,  the  research  team  developed  IMI  evaluation  criteria  derived 
from  the  scientific  literature  and  from  an  in-depth  review  of  existing  Army  IMI.  Starting  with  an 
identified  pool  of  N=  427  individual  IMI  lessons,  the  sample  was  reduced  to  N=  179  IMI 
lessons  for  in-depth  review.  The  reduction  was  based  on  selecting  IMI  that  were  relevant  to 
Combat  Arms/Maneuver,  Fires,  and  Effects  as  well  as  to  Squad/Team  Leaders.  The  IMI  we 
reviewed  came  from  various  sources,  including:  MCoE  DOTD,  Fires  Center  of  Excellence 
(FCoE)  DOTD,  the  Army  Knowledge  Online  (AKO)  My  Training  Tab  (MT2),  and  the 
Distributed  Learning  Development  Center  (DLDC),  Camp  Dodge,  IA.  The  sample  was 
evaluated  in  terms  of  structural,  pedagogic,  and  programming  characteristics  to  detennine  how 
point  of  need  and  tailored  training  techniques  could  be  applied  to  both  existing  and  future  IMI. 

Findings: 

The  broad  shift  in  educational  and  training  philosophy  represented  by  the  ALM  will 
require  a  parallel  shift  in  how  IMI  is  designed.  Existing  IMI  was  often  produced  to  reach  a 
general  audience — using  a  one-size-fits-all  design  philosophy.  Implementing  point  of  need  and 
tailored  training  on  a  large  scale  using  existing  IMI  may  be  prohibitive  in  terms  of  cost  and  time 
because  of  the  changes  in  instructional  design  philosophy  introduced  by  the  ALM.  Even  so,  we 
were  able  to  identify  some  low-cost  and  low-bandwidth  design  techniques  and  principles  that  can 
be  incorporated  into  future  IMI  to  better  adapt  content  to  Soldiers’  individual  learning  needs. 
These  involve  whole-task  assessment,  detailed  feedback  to  learners  on  their  areas  of  difficulty, 
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allowing  learners  to  make  informed  choices  about  how  to  progress  through  the  IMI,  multiple 
learning  paths,  part-task  assessment  with  appropriate  scaffolding  for  different  learners’  needs, 
and  post  training  evaluation  and  feedback  to  help  learners  understand  what  they  have  learned  and 
to  enable  them  to  plan  future  learning  tasks.  Some  design  features  that  enable  Soldiers  to  select 
different  learning  paths  have  been  incorporated  into  more  recent  Army  IMI,  such  as  ‘A  Day  in 
the  Bam.’  However,  these  implementations  often  use  complex  and  high-bandwidth  multimedia 
features,  which  can  limit  accessibility.  Moreover,  they  also  tend  to  focus  on  teaching  decision- 
action-consequence  patterns,  rather  than  focusing  on  the  essentials  of  tasks  that  tend  to  be  more 
cognitively  or  procedurally  oriented. 

Utilization  and  Dissemination  of  Findings: 

This  report  presents  our  findings  from  the  first  phase  of  a  two  phase  research  effort.  This 
report  focuses  on  our  evaluation  of  Anny  IMI  and  identification  of  learning  principles  and 
tailored  training  techniques  to  address  point  of  need.  A  subsequent  report  will  present  proof-of- 
concept  IMI  exemplars  incorporating  the  features  we  describe  here,  as  well  as  the  results  of  an 
experimental  test  of  the  IMI  exemplars. 
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Addressing  Point  of  Need  in  Interactive  Multimedia  Instruction: 

A  Conceptual  Review  and  Evaluation 

Introduction 

In  201 1,  The  Army  Learning  Concept  2015  (TRADOC  PAM  525-8-2)  was  published, 
articulating  a  new  learning  model  for  Anny  training  and  education  (TRADOC,  2011a).  The  new 
Army  Learning  Model  (ALM)  introduced  point  of  need  as  a  concept  to  guide  the  design  and 
deployment  of  Army  learning  resources  in  order  to  better  meet  Soldiers’  various  learning  needs 
and  professional  requirements.  Point  of  need  speaks  to  the  many  reasons  Soldiers  have  for  using 
Army  learning  resources;  for  example,  they  may  want  to  strengthen  a  skill  diminished  by  time  or 
they  have  encountered  a  situation  that  demands  they  acquire  a  new  skill.  The  central  feature  of 
point  of  need  training  is  its  specificity.  Point  of  need  training  seeks  to  address  a  particular 
learning  need  for  a  particular  audience  at  a  particular  time.  This  is  in  contrast  to  an  approach  that 
seeks  to  develop  training  that  may  be  used  to  reach  the  broadest  audience  possible. 

As  one  type  of  Army  learning  resource,  which  is  fairly  common,  interactive  multimedia 
instruction  (IMI)  can  provide  Soldiers  nearly  immediate  access  to  learning  material  wherever  and 
whenever  their  need  arises.  It  may  also  be  adapted  to  individual  Soldiers’  learning  needs.  This 
research  explored  ways  to  address  Soldiers’  specific  learning  needs  and  professional 
requirements  by  incorporating  various  design  features  and  learning  principles  into  existing  and 
future  IMI  in  order  to  emphasize  a  point  of  need  approach. 

Within  the  context  of  Anny  training,  IMI  has  been  used  to  address  a  variety  of  learning 
needs,  such  as  (a)  developing  critical  skills  Soldiers  need  before  entering  courses,  (b)  supporting 
instruction  within  courses,  (c)  refreshing  knowledge  and  skills  related  to  particular  topics  after 
graduating  from  courses,  as  well  as  (d)  providing  comprehensive  training  materials  covering 
complete  courses  (Straus,  Shanley,  Bums,  Anish,  &  Crowley,  2009).  Point  of  need  is  intended 
to  expand  the  capabilities  of  IMI  by  specifying  a  constellation  of  requirements  which  indicate 
that  questions  concerning  how  and  what  is  trained  are  as  important  as  where  and  when  it  is 
trained.  Anny  Learning  Concept  (ALC)  2015  states,  “[ljeaming  is  best  achieved  at  the  point  of 
need  and  therefore  must  be  accessible  in  a  career-long  learning  continuum,  rather  than  limited  to 
specific  timeframes  or  locations”  (TRADOC,  2011a,  p.  7).  This  statement  implies  that:  (a) 
learners  should  be  able  to  access  a  learning  resource  wherever  and  whenever  it  is  needed,  and  (b) 
that  the  accessed  learning  resource  is  tailored  to  learners’  current  level  of  understanding  and 
presents  to  them  what  they  need  to  know  to  accomplish  a  specific  job-related  task  and/or 
mission. 

Point  of  need  training  can  also  be  considered  in  terms  of  the  depth  and  breadth  of  the 
information  being  presented  and  the  purpose  of  the  instruction.  The  concept  may  connote  bite- 
sized  chunks  of  content,  designed  to  address  only  a  very  specific  learning  need,  when  a  domain 
does  not  require  great  detail  and  depth  of  infonnation  to  train  properly.  It  may  also  point  to  more 
in-depth  training  when  necessitated  by  the  content  domain  and  the  needs  of  the  learner.  What 
point  of  need  seeks  to  do  is  to  define  the  specific  needs  of  the  learner  and  target  those  learning 
needs.  When  only  considering  point  of  need  in  tenns  of  the  duration  of  training,  we  may  not 
fully  consider  the  complexity  of  the  topic  we  are  seeking  to  train. 
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One  way  to  address  point  of  need  is  to  explore  design  features  and  learning  principles 
that  may  be  applied  to  structure  the  IMI  learning  experience.  Careful  application  of  these 
principles  can  help  in  developing  IMI  that  can  address  learners’  needs  at  different  times  and 
places  and  with  respect  to  the  particular  depth  of  knowledge  they  seek.  To  date,  most  Army 
research  on  IMI  has  focused  on  how  to  improve  the  features  of  courseware,  such  as  its  technical 
characteristics,  production  quality,  or  mode  of  delivery.  Less  research  has  focused  on  pedagogy 
or  on  applying  tailoring  techniques  within  IMI  to  better  address  individualized  learning  needs 
(see  Giglio,  2009;  Straus  et  al.,  2009;  Straus  et  ah,  2011). 

To  address  point  of  need  requires  taking  a  different  perspective  toward  IMI — that  is,  a 
perspective  that  first  emphasizes  Soldiers’  reasons  for  using  the  IMI,  and  second,  the 
technological  characteristics  of  the  IMI.  For  the  purposes  of  this  research,  we  considered  three 
point  of  need  variations.  First,  some  Soldiers  may  need  to  know  the  basic  steps  to  complete  a 
task,  i.e.,  the  core  of  a  task.  This  type  of  training  would  have  narrowly  scoped  content,  but 
sufficient  depth  of  information  to  execute  a  task.  The  point  of  need  with  core  training  would 
tend  to  be  more  procedural  in  nature,  as  in  Call  for  Fire  or  Adjust  Fire  tasks.  The  training 
materials  would  cover  the  basic  steps  involved  in  executing  the  task,  with  supplemental 
information  provided  if  necessary  to  support  task  execution.  Other  Soldiers  may  only  need  an 
overview  of  basic  infonnation  for  a  topic  or  skill,  i .q.,  familiarization  with  the  subject  domain. 
This  type  of  training  would  be  broad  in  scope,  but  have  less  depth  of  infonnation  than  core 
training.  Familiarization  training  would  tend  to  address  key  concepts  and  more  evaluative  tasks, 
such  as  those  involved  in  detennining  squad  defensive  positions  in  an  urban  environment.  Still, 
other  Soldiers  may  already  have  a  good  base  of  knowledge  and  skill,  so  the  IMI  must  be  able  to 
address  difficulties  that  arise  for  them  as  individuals,  i.e.,  incorporating  tailored  training 
principles.  A  tailored  training  approach  needs  to  be  flexible  with  respect  to  the  breadth  and 
depth  of  information  being  presented.  In  other  words,  a  tailored  training  approach  may 
incorporate  both  core  and  familiarization.  In  addition,  it  would  apply  techniques  to  assist 
learners  in  being  aware  of  their  own  areas  of  difficulty  and  helping  them  to  make  informed 
decisions  concerning  how  to  proceed  through  their  training.  Together,  these  types  of  point  of 
need  training  likely  cover  a  large  portion  of  situations  in  which  Soldiers  would  need  learning 
materials  to  address  their  specific  infonnation  or  training  needs,  whether  those  needs  concern 
new  learning  or  refreshing  existing  knowledge  and  skills. 

In  order  to  design  IMI  to  best  meet  a  variety  of  needs,  it  should  be  able  to  address 
multiple  levels  of  comprehension  and  experience,  as  well  as  meet  Soldiers’  present  expectations 
and  professional  requirements.  For  instance,  Dyer,  Singh,  and  Clark  (2005)  found  that  One 
Station  Unit  Training  (OSUT)  Soldiers  tended  to  require  training  that  is  more  highly  structured 
than  did  Infantry  Officer  Basic  Course  Soldiers,  who  benefited  more  from  being  able  to  control 
how  and  what  they  were  learning.  Learning  resources  designed  to  address  point  of  need  should 
incorporate  features  that  allow  the  learner  to  exert  varying  degrees  of  control  over  what  he  or  she 
is  learning — an  approach  that  accords  with  the  ALM’s  learner-centric  perspective.  The  ALM 
point  of  need  concept  presents  a  vision  of  future  IMI  that  may  better  address  individual 
differences  in  knowledge  and  skills  and  in  desired  learning  outcomes  (TRADOC,  2011a).  As 
Dyer,  Singh,  and  Clark  (2005)  noted:  “careful  thought  should  be  given  to  computer  based 
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training  design  to  account  for  individual  differences  and  to  maximize  learning  on  the  part  of  all” 
(pg.  33). 

Interactive  Multimedia  Instruction  at  the  Point  of  Need 

Although  it  is  a  concept  finding  new  application  in  ALM,  point  of  need  has  been  studied 
in  a  variety  of  contexts  to  include  library  science  (Trump  &  Tuttle,  2001;  Lipow,  2003;  Walsh, 
2010),  instructional  design  (Brown,  1997),  and  medical  training  (Harun,  2002),  among  others. 
Walsh  (2010)  applied  Global  Positioning  System  technology  to  provide  library  users  with 
resource  recommendations  based  on  their  location  within  a  library  and  the  topic  of  their  search; 
the  user’s  point  of  need  was  an  infonnational  need  that  was  addressed  in  relation  to  where  they 
were  within  the  library.  Brown  (1997)  conceptualized  point  of  need  in  terms  of  accessibility  of 
internet  content,  i.e.,  the  anywhere/anytime  view  of  point  of  need  that  was  prevalent  in  the 
1990s.  However,  she  also  drew  in  elements  related  to  students  self-pacing  within  online 
instruction,  exploring  topics  of  interest  in  greater  depth,  and  skipping  over  topics  they  felt  were 
already  understood  well.  Finally,  in  the  context  of  medical  training,  Harun  (2002)  described 
point  of  need  as  a  targeted  application  of  a  limited  amount  of  infonnation;  just  what  the  learner 
needs  to  know  given  his  or  her  present  activity.  The  ALM  point  of  need  concept  seems  to  cover 
the  breadth  of  concerns  that  emerge  in  previous  research  on  point  of  need,  as  it  focuses  both  on 
factors  related  to  time  and  place — a  delivery  concern — and  on  tailoring  to  specific  learners’ 
needs — a  training  and  education  concern  with  design  implications. 

In  a  recent  video,  Army  Learning  Concept  2015:  Thinking  Soldiers  -  Learning  Army! 
(TRADOC,  2010),  TRADOC  provided  striking  examples  of  what  it  envisions  for  IMI  designed 
to  address  point  of  need.  In  the  first  example,  a  young  infantry  Soldier  is  accessing  point  of  need 
IMI  on  room  clearing  operations,  using  a  personal  digital  device.  His  fire  team  is  conducting 
collective  training  using  precision  room  clearing  techniques.  He  reviews  these  materials 
immediately  prior  to  joining  his  fire  team  in  live  training.  In  the  second  example,  a  medical 
specialist  is  en  route  to  an  incident  on  an  aero  medical  evacuation  helicopter.  The  Soldier 
accesses  IMI  for  medical  treatment  procedures  on  her  personal  digital  device  before  she  arrives 
at  the  site.  When  she  arrives  she  moves  quickly  employing  her  recently  refreshed  skills  to  treat 
casualties. 

These  examples  vividly  illustrate  how  the  Army  anticipates  applying  digital  technologies 
to  accomplish  aspects  of  point  of  need  training  as  the  ALM  is  implemented.  Digital  learning 
resources  can  be  effective  and  efficient  tools  to  address  requirements  of  accessing  and  leveraging 
training  wherever  and  whenever  it  is  needed.  Also,  given  that  digital  learning  resources  are  often 
completed  by  individual  learners,  they  provide  a  means  for  addressing  a  critical  aspect  of  the 
point  of  need  concept:  tailoring  training  to  fit  individual  Soldiers’  needs. 

Impact  of  IMI  within  the  context  of  the  ALM 

IMI  that  can  address  point  of  need  is  significant  to  supporting  TRADOC ’s  goal  of  having 
Soldiers  and  leaders  who  are  “fully  prepared  to  prevail  in  complex,  uncertain  environments” 
(ALC  2015,  TRADOC,  2011a,  p.  19).  While  the  ALM  is  changing  how  digital  media  and  digital 
technologies  are  being  used  in  the  Anny,  it  may  also  initiate  changes  in  organizational  practices 
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to  support  the  emerging  learning  environment.  For  instance,  Straus  et  al.  (2011)  found  that 
training  policy,  training  circumstances,  and  student  characteristics  were  often  the  most 
significant  factors  related  to  long-tenn  knowledge  retention.  Point  of  need  IMI  can  support 
implementation  of  ALM’s  learner-centric  environment  across  three  main  learning  contexts  (a) 
context-based,  collaborative,  problem-centered  instruction,  (b)  blended  learning,  and  (c) 
distributed  learning  (dL).  IMI  will  need  to  be  adapted,  either  by  design  or  by  application  to  fit 
these  various  learning  contexts. 

Becoming  learner-centric.  Soldiers  who  spent  more  time  working  on  IMI  retained  more 
knowledge  over  time  (Straus  et  al.,  2011).  It  is  noteworthy  that  the  time  Soldiers  are  able  to 
invest  using  IMI  for  learning  is  often  under  the  control  of  their  commander.  Straus  et  al.  noted 
that  commanders  who  permitted  their  Soldiers  to  use  duty  time  for  study,  review,  and  course 
preparation,  also  had  Soldiers  who  spent  more  time  using  IMI  materials  to  leam.  Soldiers  who 
were  only  able  to  use  IMI  on  their  own  time  used  IMI  less,  and  retained  less  knowledge.  A  clear 
implication  of  this  finding  is  to  shorten  IMI  modules  so  they  can  be  completed  within  periods  of 
opportunity  during  the  duty  day  to  enable  learning  activities  to  be  interspersed  with  other  duties. 

The  future  learner-centric  environment  must  provide  context-based,  collaborative,  and 
problem-centered  instruction.  These  characteristics  are  intended  to  reduce  the  Army’s  reliance 
on  instructor-centered  lectures,  and  replace  it  with  more  learner-centered  instructional 
methodologies.  This  shift  in  instructional  methodology  may  include  self-paced,  technology- 
delivered  instruction  outside  the  classroom  and  may  take  several  forms,  including  IMI  and  other 
types  of  dL  such  as  mobile  applications  (apps),  gaming,  wikis,  blogs,  social  media  postings  (e.g. 
Facebook),  or  other  media  that  allow  creation  and  exchange  of  user-generated  content. 

Within  a  learner-centric  environment  of  this  type,  learners  may  use  IMI  to  develop  a 
foundation  for,  be  introduced  to,  or  familiarized  with  new  knowledge  and  skills.  Depending  on 
the  intent  of  training  designers  and  the  identified  needs  of  students,  applying  the  point  of  need 
concept  may  not  always  require  materials  to  be  developed  that  go  into  great  depth  on  any  single 
skill.  This  IMI  could  instead  seek  to  give  learners  an  understanding  of  the  breadth  of  concepts 
and  tasks  involved  in  a  domain,  and  the  knowledge  of  where  and  how  to  seek  out  supplemental 
infonnation  to  facilitate  them  in  seeking  greater  depth  of  understanding  at  a  later  time.  This  IMI 
would  be  designed  to  familiarize  learners  with  a  topic  domain. 

Point  of  need  IMI  could  also  be  used  to  refresh  or  reinforce  existing  knowledge — as  is 
presented  in  the  TRADOC  video,  ‘Thinking  Soldiers,  Learning  Anny!  ’  The  IMI  could  be 
designed  to  first  assess  a  Soldier’s  current  proficiency  and  then  target  the  learning  content  to 
address  identified  deficiencies.  With  respect  to  IMI  design,  using  scenario  based  assessments 
with  detailed  feedback,  systematically  varying  training  activities  within  the  instructional  design, 
and  applying  tailored  training  techniques  to  target  user  preferences  and  knowledge  and  skill 
levels  may  support  the  learner-centered  ALM  approach  (see  Schwartz,  Brophy,  Lin,  & 

Bransford,  1999). 

Blending  IMI  with  residential  courses.  As  not  all  IMI  is  used  by  learners  outside  of  a 
residential  course,  applications  of  the  point  of  need  concept  to  IMI  used  in  blended  learning 
contexts  should  also  be  considered.  Blended  learning  has  been  most  frequently  defined  as  a 
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combination  of  online  or  technology-delivered  instruction  and  face-to-face  instruction  (see 
Graves  &  Bickley,  2009).  However,  blended  learning  is  more  than  an  amalgam  of  a  good 
trainer,  a  cooperative  peer  group,  and  IMI.  Blended  learning  is  intended  to  merge  the 
efficiencies  and  effectiveness  of  self-paced,  technology-delivered  instructional  activities  with 
those  of  the  expert  facilitator.  It  can  also  harness  the  benefits  of  peer-to-peer  interaction  within  a 
small  group  (Romiszowski,  1988).  Blended  learning  is  intended  to  leverage  the  strengths  of 
digital-age  learners  and  engage  them  through  the  use  of  multimedia  and/or  game-based 
scenarios,  while  still  retaining  the  benefits  of  face-to-face  discussions  and  knowledge  sharing 
(Jones  &  Bronack,  2007).  IMI  applications  used  in  a  blended  learning  setting  are  likely  best 
designed  to  be  less-structured  and  focused  around  collaborative  learning  activities.  This  type  of 
design  would  better  allow  the  Army  to  avail  itself  of  the  advantages  of  the  face-to-face  learning 
situation,  while  also  incorporating  learning  technologies  to  support  learning. 

Blended  learning  may  also  potentially  be  used  to  decrease  learning  times.  Research  in 
the  use  of  blended  learning  indicates  that  students  may  attain  a  30%  decrease  in  the  time  that  it 
takes  to  leam  with  no  decrease  in  effectiveness  (see  Fletcher  and  Chatham  2009;  as  cited  in 
TRADOC,  2011a).  The  “guide  on  the  side”  instructor  requires  increased  proficiency  in 
communications  skills,  needs  increased  mastery  of  the  associated  knowledge  and  skills,  and  must 
be  skilled  in  the  incorporation  and  use  of  appropriate  media  and  technologies  throughout 
training.  It  should  also  be  noted  that  increased  efficiency  seems  to  be  attained  most  often  when 
the  technology-delivered  instruction,  IMI  or  other  forms  of  dL,  is  designed  employing 
established  learning  principles  coupled  with  appropriate  learning  content  (Fletcher  &  Chatham, 
2009). 


Distributed  learning  and  accessibility.  Finally,  IMI  is  expected  to  play  a  key  role  in  all 
future  Anny  life-long  learning  models  (TRADOC,  2011a).  One  critical  reason  for  this  is  its 
accessibility.  Through  its  varied  forms,  IMI  provides  the  capability  to  deliver  learning  content  at 
the  point  of  need,  when  and  where  the  knowledge  and  skills  are  required  to  be  introduced, 
refreshed,  enhanced,  practiced,  or  mastered.  However,  to  meet  this  challenge,  IMI  products  must 
be  up-to-date,  easily  accessible,  and  engaging.  Content  available  to  the  Soldier  must  be 
expanded,  broader  and  more  inclusive  of  a  wider  variety  of  experience,  tasks,  skills,  and 
knowledge  levels.  The  content  of  IMI  products  must  be  tailorable  to  the  learner’s  current  level 
of  understanding  and  needs.  For  this  to  be  possible  the  learner’s  needs  and  requirements  must  be 
accurately  assessed,  and  the  training  delivered  must  be  tailored  to  meet  the  Soldier  where  he  or 
she  is. 


While  the  implementation  of  ALM  is  well  underway,  it  has  varied  across  TRADOC. 
Some  institutional  courses  are  already  learner-centric,  having  implemented  many  of  the  ALM 
adult  learning  concepts  and  principles,  such  as  facilitation  and  collaborative  learning.  Recent 
research  has  indicated  that  a  number  of  courses  are  already  applying  tailored  training  to  address 
individual  or  small  group  student  needs.  Even  so,  many  other  courses — even  those  that  exhibit 
some  ALM  learner-centric  characteristics — seemed  to  remain  centered  on  trainers  and 
institutions,  rather  than  on  students  (Dyer,  Wampler,  &  Blankenbeckler,  2011). 

It  is  a  similar  situation  with  respect  to  IMI.  Some  IMI  was  developed  well  before  ALM 
began  to  be  implemented.  This  older  IMI  exhibits  design  characteristics  that  tend  to  be 
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standardized  and  sequential.  More  recent  IMI  has  sought  to  incorporate  more  interactive  and 
media-rich  features,  many  with  less  sequential  narrative  structures  such  as  branching  storylines. 
Yet,  even  this  current  IMI  is  not  necessarily  designed  to  target  specific  learners’  needs,  even  if 
the  learner  can  interact  with  the  media.  As  ALM  is  implemented,  IMI  will  likely  continue  to 
develop  in  tandem  with  institutional  training  as  they  become  ever  more  blended  in  schoolhouse 
courses. 

Stand-alone,  computer-based  learning  via  the  internet  has  supported  Anny  initiatives 
focused  on  life-long  learning,  self-development,  as  well  as  knowledge/skill  acquisition  and 
maintenance  (Graves,  Rauchfuss,  &  Wisecarver,  2012).  It  is  clear  that  the  internet  will  continue 
to  be  relied  on  as  a  tool  to  deliver  information  and  training  materials.  In  recent  years,  internet 
access  has  expanded  from  desktop  computers  to  also  include  mobile  devices,  such  as  smart¬ 
phones,  tablets,  and  digital  readers.  In  the  future,  the  cloud  and  other  approaches  to  centralized 
data  management  and  delivery  will  continue  to  support  development  of  ever  higher  quality,  more 
immediately  accessible  and  up-to-date  learning  resources. 

Although  the  ALM  is  part  of  a  Campaign  of  Learning  that  could  potentially  revolutionize 
how  the  Army  trains  Soldiers  and  leaders,  it  is  a  slow-going  revolution.  The  ALM  is  being 
implemented  in-stride  (TRADOC,  2011a),  and  many  simultaneous  steps  toward  implementation 
are  underway.  For  IMI,  Army  Knowledge  Online  (AKO)  now  provides  enhanced  access  to  the 
Army  Training  (and  Education)  Network  (ATN).  In  late  February  2012,  the  Army  migrated 
approved  IMI  courses  and  other  training  materials  to  the  MyTraining  Tab  (MT2)  of  the  ATN,  as 
a  step  toward  creating  a  “single  portal  to  digital  resources.”  To  support  their  training  programs 
and  as  a  transitional  step,  many  Anny  Training  CoEs  already  provide  access  to  training 
materials,  training  references,  and  IMI  courseware. 1  It  seems  that  broad  accessibility  is  already 
addressing  the  wherever/whenever  needed  aspect  of  point  of  need. 

With  the  vast  array  of  learning  resources  being  developed  and  currently  available, 
Soldiers  need  a  way  to  manage  their  individual  learning  initiatives  (Graves,  Rauchfuss,  & 
Wisecarver,  2012).  Straus  et  al.  (2011)  reported  that  Soldiers  learning  success  was  increased  by 
having  a  shorter  lag  time  between  completing  the  IMI  phase  of  a  course  and  starting  the 
residential  phase.  It  would  seem  that  consistent,  focused  learning  needs  to  be  spaced 
appropriately  to  enhance  Soldiers’  learning  and  longer  tenn  retention  of  knowledge  and  skills. 
Soldiers  may  be  supported  by  tools  like  the  Army  Career  Tracker  to  find  the  optimal  time  to 
complete  IMI  prior  to  beginning  a  residential  course,  or  to  follow  a  long-term  learning  path 
(Gary  Rauchfuss,  personal  communication,  23  April,  2013). 

Addressing  Problems  with  IMI  Reuse/Repurposing 


1  The  IMI  courseware  available  on  CoE  sites  includes  materials  not  yet  migrated  to  the  ATN  site,  such  as  IMI  that  is 
in  development  or  not  yet  posted  to  the  ATN  catalog  and  library  or  held  for  use  in  proponent-controlled  functional 
or  leader  courses.  For  example,  a  CoE-improved  access  portal  can  be  found  on  the  Fires  Center  of  Excellence 
(FCoE)  Reach  Back/Reset  Training  site  at  https://firescoe.sill.armv.mil/.  The  FCoE  site  provides  access  to  an 
extensive  resource  library  of  training  materials  and  references  for  all  Field  Artillery  (FA)  and  Air  Defense  Artillery 
(ADA)  military  occupational  specialties  and  training  courses. 
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Another  significant  issue  that  has  arisen  in  the  development  and  deployment  of  Anny 
IMI  is  that  of  reuse.  In  our  research,  we  considered  the  feasibility  of  reusing/repurposing 
existing  IMI  to  better  fit  the  ALM  learner-centric  design  philosophy  and  learners’  specific  needs. 
In  historical  context,  the  intent  to  reuse  IMI  has  been  integral  to  the  development  process.  In 
January  of  1999,  the  Executive  Branch  of  the  United  States  (U.S.)  Government  recognized  the 
impact  of  the  computer  on  training  and  acknowledged  a  revolution  in  learning  when  President 
Clinton  signed  Executive  Order  13111  (Clinton,  1999).  Key  tenants  of  this  directive  were  the 
mandates  to  “promote  and  integrate  the  effective  use  of  training  technologies,”  establishment  of 
a  mechanism  for  development  of  standards  for  training  software,  and  a  desire  for  both  efficiency 
and  economy  through  the  promotion  of  “reusable  training  component  software.”  The 
Department  of  Defense  (DoD)  had  taken  steps  two  years  earlier  through  the  Office  of  the 
Undersecretary  of  Defense  for  Personnel  and  Readiness  to  establish  the  Advanced  Distributed 
Learning  (ADL)  initiative  (Fletcher,  2010).  A  stated  goal  of  the  ADL  initiative  was  to  assist  the 
Armed  Forces  to  make  learning  available,  on  demand,  anytime  and  anyplace. 

Army  and  DoD  interest  in  the  usability  and  reuse  of  software  components,  including 
elements  of  IMI,  can  be  traced  to  the  late- 1980s  and  lessons  drawn  from  the  European,  Japanese, 
and  U.S.  commercial  software  industry.  These  commercial  organizations  had  demonstrated 
economy  and  efficiency  with  the  reuse  of  existing  software.  Additionally,  they  had 
demonstrated  significant  potential  for  increasing  engineering  productivity  and  rapid  marketing  of 
new  software  products  and  version  improvements,  while  maintaining  high  system  quality  as  well 
as  decreasing  the  cost  while  building  and  maintaining  large,  software-intensive  systems 
(Software  Productivity  Consortium  Services  Corp.,  1993). 

Indeed,  some  early  goals  of  ADL  seem  to  be  coming  to  fruition.  With  the  broad 
availability  of  miniaturized  computing  platforms  in  the  form  of  small  robust  tablets  and  highly 
capable  smart  cellular  telephones,  the  envisioned  portable  user  interface  devices  are  reality.  The 
capabilities  and  features  of  these  platfonns  seem  to  expand  exponentially  as  new  models  are 
rapidly  introduced  into  the  market.  The  standardization  and  structure  of  training  media  directed 
by  the  requirements  of  the  Sharable  Content  Object  Reference  Model  (SCORM)  (DoD,  2011) 
provides  that  components  of  dL  materials  developed  for  the  military  audiences  will  be  structured 
to  be  portable,  durable,  and  reusable.  These  components  are  identified  as  Sharable  Content 
Objects  (SCOs).  A  SCO  represents  the  lowest  level  of  granularity  of  a  learning  resource  that  a 
learning  management  system  (LMS)  should  track.  The  SCORM  requirements  do  not  impose  any 
particular  constraints  or  minimum  requirement  on  the  size  of  a  SCO.  A  SCO  may  be  a  single 
graphic  on  a  single  web  page,  or  it  can  be  a  collection  of  web  pages.  However,  a  SCO  should  be 
a  single  unit,  independent  of  its  learning  context  and  reusable  in  a  similar  or  new  context.  To 
achieve  such  reusability,  a  SCO  should  be  self-contained  and  not  reference  or  link  to  other 
SCOs.  SCOs  should  be  available  through  the  source  files  of  their  parent  courseware. 

The  implementation  of  a  Content  Object  Repository  Registration/Resolution  Architecture 
(CORDRA)  for  cataloging  SCOs  in  the  ADL  registry  may  soon  provide  improved  visibility  to 


2  SCORM  is  a  standard  for  a  technical  framework  to  enable  the  use  of  Web-based  e-Learning  content  across 
multiple  environments  (e.g.,  LMSs).  SCORM  defines  how  individual  instruction  elements  are  combined  at  a 
technical  level  and  sets  conditions  for  the  software  needed  to  use  the  content.  For  further  explanation,  see  “SCORM 
2004,  3rd  Edition"  (Advanced  Distributed  Learning,  2008). 
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more  readily  facilitate  reuse  or  repurposing  of  SCOs.  With  these  advances,  it  would  seem  that 
with  the  addition  of  an  improved  network  and  an  intelligent  tutor,  Soldiers  should  soon  be  able  to 
engage  in  training  and  learning  tailored  to  their  wants  and/or  needs  at  any  time  and  place. 

Indeed,  Hu,  Graesser,  and  Fowler  (2010;  as  cited  in  Fletcher,  2010),  envision  a  future 
environment  in  which  the  user,  through  a  smart  device,  will  access  tailored  learning.  They  argue 
that  this  access  and  learning  would  not  only  be  through  pre-specified  lessons  (existing,  purpose- 
built  IMI),  but  in  a  manner  akin  to  the  age  old  tutorial  practice  of  a  student  and  a  mentor  working 
together  to  enhance  knowledge  and  refine  the  skills  of  the  student.  What  differs  in  the  scenario 
they  envision  is  that  the  human  mentor  would  be  replaced  by  a  computational  device — 
accessing,  selecting,  organizing,  and  delivering  SCOs  drawn  from  a  vast  repository  through  a 
networked  infonnation  infrastructure.  Additionally,  these  smart  devices  acting  in  a  system  of 
systems  would  also  support  assessment  of  student  progress,  determine  the  need  for  prerequisite 
or  foundation  knowledge,  certify  accomplishments,  and  forecast  skill  and  knowledge  decay  to 
support  retraining.  The  knowledge  management  system  would  trigger  remediation,  retraining, 
reinforcement  or  scaffolding  at  appropriate  times,  as  well  as  provide  progressively  more 
advanced  training  to  improve  knowledge,  refine  skills,  or  respond  to  the  student’s  desire  for 
exploration  of  new  knowledge. 

The  Continuing  Challenges  of  Reuse  and  Repurposing  of  IMI,  Many  of  the  modular 
approaches  to  IMI  development  and  delivery  were  challenged  by  Shanley  et  al.  (2009)  in  a  study 
of  the  challenges  of  IMI  reuse  for  the  Army  Distributed  Learning  Program.  Shanley  et  al.  found 
that  significant  savings  or  returns  from  dL  reuse  are  exceptions.  In  fact,  only  25%  of  the  training 
development  organizations  estimated  positive  return  on  investment  through  reuse.  Most 
organizations  recognized  lower  than  expected  returns  and  35%  reported  no  savings  or  a  loss  (i.e. 
time,  labor,  dollars).  Most  organizations  noted  significant  technical  issues,  unexpected 
problems,  and  complexities  of  reuse.  Some  of  the  obstacles  reported  include:  issues  with 
metadata  or  repositories,  lack  of  strategic  planning  for  reuse,  cultural  issues  blocking 
implementation  (e.g.,  “not  invented  here”  attitude),  legal  or  security  issues,  lack  of  training  on 
tools  needed  for  repurposing  materials,  failure  of  collaboration  between  agencies,  difficulty 
changing  established  design  processes,  and  difficulty  defining  what  constituted  a  SCO.  Given 
these  concerns,  some  organizations  felt  that  it  was  easier,  cheaper,  or  more  efficient  to  create 
new  content  rather  than  find,  review,  validate,  and  repurpose  existing  learning  objects/SCOs. 

Even  though  organizations  confonned  with  SCORM  requirements  for  developing 
instructional  content,  the  lack  of  firm  defining  criteria  for  a  SCO  led  to  numerous  challenges 
when  trying  to  reuse  dL.  Shanley  et  al.  (2009)  reported  that  most  successes  with  reusable 
learning  objects/SCOs  typically  involved  the  repurposing  of  modules  or  larger  chunks  of  content. 
The  examples  provided  by  training  development  organizations  indicated  that  modules  were  taken 
from  existing  courses  and  repurposed  into  smaller  stand-alone  learning  packages  in  order  to 
reach  larger  audiences.  However,  participating  organizations  described  greater  success  with 
reuse  of  entire  courses  instead  of  reusable  learning  objects/SCOs  or  modules  and  chunks.  When 
organizations  decided  to  reuse  courseware,  they  often  recognized  that  the  courses  and  learning 
materials  were  of  interest  across  student  populations  and  courses.  Frequently  this  reuse  of 
courseware  required  running  the  courseware  on  a  new  LMS.  The  transition  was  often  far  from 
transparent.  Fine  adjustments  were  almost  always  required  to  make  these  reused  learning  objects 
compatible  with  the  new  LMS. 
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In  addition  to  examining  the  current  state  of  reuse,  Shanley  et  al.  (2009)  also  examined 
trends  that  could  forecast  or  indicate  a  probability  of  increased  reuse.  The  study  reviewed 
industry  and  trade  information  on  the  demand  for  and  sales  of  learning  content  management 
systems  (LCMS).  LCMS  software  provides  training  developers  with  authoring  applications,  data 
repositories,  delivery  interfaces,  and  a  variety  of  administrative  tools  to  aid  in  the  management  of 
e-Leaming  content.  Low  sales  and  demand  data  for  available  LCMS  seem  to  indicate  that  few 
organizations  are  actively  pursuing  or  planning  extensive  object  reuse.  Organizations  seemed 
disinclined  to  invest  in  the  required  tools  for  future  reuse  efforts. 

Rationale  and  Methodology 

Given  that  little  research  has  investigated  applications  of  the  point  of  need  concept  in 
Army  IMI,  we  undertook  this  research  effort  to  explore  these  potential  applications.  We  sought 
to  determine  possible  ways  point  of  need  could  be  applied  to  existing  IMI  as  well  as  to  the 
development  of  future  IMI.  The  objectives  of  this  research  were  to: 

•  Review  existing  Anny  IMI  developed  for  the  Maueuver,  Fires,  and  Effects  (MFE) 

domain  in  order  to  identify  IMI  that  could  address  the  specific  needs  of  new 

Squad/Team  Leaders;3 

•  Evaluate  the  characteristics  of  selected  IMI  for  modification  to  three  point  of  need 

variations:  familiarization,  core,  and  tailored  training;  and 

•  Propose  techniques  to  address  applications  of  point  of  need  in  reusing  existing  IMI  and  in 

developing  future  IMI. 

While  conceptual,  organizational,  and  technical  barriers  can  disrupt  the  process  of 
reusing  or  modifying  existing  IMI,  we  sought  to  review  existing  Army  IMI  to  determine  if  it 
could  be  modified  to  address  point  of  need  as  well  as  to  identify  design  features  and  tailored 
training  techniques  that  could  potentially  be  incorporated  into  existing  and/or  future  IMI.  The 
focus  of  the  current  report  is  conceptual  and  evaluative,  reporting  on  the  work  we  did  prior  to 
beginning  development  of  point  of  need  IMI  exemplars.  The  IMI  exemplars  and  the  results  of 
experiments  testing  the  different  point  of  need  variations  will  be  presented  in  a  subsequent 
report.  This  report  focuses  on  our  review  and  evaluation  of  current  Anny  IMI,  and  the  design 
features  and  tailored  training  techniques  we  view  as  amenable  to  a  point  of  need  IMI  training 
context.  The  results  reported  here  emerged  from  lessons  we  learned  when  investigating 
candidate  IMI  courses  for  reuse/modification  to  various  point  of  need  fonnats. 

Identifying  IMI  for  Evaluation 

For  the  purposes  of  this  report,  we  used  the  following  definitions  to  identify  different 
levels  of  courseware  components: 


3  We  limited  our  review  to  IMI  relevant  to  Maneuver,  Fires,  and  Effects,  and  Squad/Team  Leaders  in  order  to  make 
the  review  executable.  A  census-type  evaluation  of  all  Army  IMI  would  have  been  too  broad-ranging  to  execute 
effectively. 
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•  Course  -  An  entire  block  of  training  that  contains  all  materials  and  content  for  the  given  subject 
area.  A  course  may  contain  multiple  lessons,  tests,  reference  materials,  navigation  instructions, 
etc.  As  an  example,  a  course  covering  the  subject  area  of  “map  reading”  may  contain  multiple 
lessons  consisting  of  plotting  grid  coordinates,  colors  on  the  map,  topographic  features,  legend, 
etc. 

•  Lesson  -  A  block  of  instruction  that  covers  a  specific  subject,  not  a  broad  subject  area.  In  many 
Army  courses,  a  lesson  would  cover  a  specific  task,  such  as  “plot  grid  coordinates,”  or  “identify 
map  symbols  from  the  legend.”  The  lesson  would  consist  of  the  training  material,  could  contain 
checks  on  learning,  or  a  test  to  assess  learning  for  that  specific  subject. 

•  Module  -  Within  a  lesson,  instructional  material  is  frequently  organized  and  grouped  into  smaller 
chunks,  which  we  consider  to  be  modules.  Each  module  is  generally  a  small  chunk  of  learning 
that  covers  the  learning  objectives  and  possibly  includes  a  quiz  or  check  on  learning.  The  module 
could  be  considered  the  smallest  stand-alone  block  of  instructional  material. 

•  Sharable  Content  Object  (SCO)  -  IMI  developers  have  inteipreted  SCO  to  mean  many  different 
things.  In  some  instances,  SCO  refers  to  merely  a  single  graphic  or  an  audio  file  attached  to  an 
instructional  page,  while  in  other  cases  it  could  mean  several  pages  of  content.  Here,  we  defined 
a  SCO  as  any  piece  of  IMI  that  is  smaller  than  a  self-contained  chunk  of  learning. 

Given  that  a  comprehensive  review  of  all  Anny  IMI  was  not  feasible,  we  worked  with  the 
Directorates  of  Training  and  Doctrine  (DOTDs)  at  the  Maneuver  Center  of  Excellence  (MCoE), 
Fort  Benning,  GA,  and  the  Fires  Center  of  Excellence  (FCoE),  Fort  Sill,  OK,  to  obtain  IMI  for 
review.  In  addition,  we  identified  other  IMI  available  on  the  Army  Knowledge  Online  (AKO) 
My  Training  Tab  (MT2)  site.  While  obtaining  IMI  was  an  essential  aspect  of  our  review,  access 
to  source  files  containing  original  graphics,  text  files,  reference  materials,  and  narrations  was 
critical  to  explore  how  we  could  repurpose  and  reuse  the  materials  to  address  point  of  need.  The 
MCoE  and  FCoE  DOTDs,  as  well  as  the  Distributed  Feaming  Development  Center  (DFDC), 
Camp  Dodge,  IA,4  provided  available  files  for  selected  materials.  Additionally,  to  assist  in 
detennining  tangible,  point  of  need  targets  for  training,  the  MCoE  provided  a  list  of  29 
individual  tasks  that  represent  competencies  required  for  small  unit  leaders.  These  tasks  ranged 
from  operator  maintenance  tasks  such  as  “Maintain  a  240B  Machine  Gun”  to  complex  leader 
tasks  such  as  “Troop  heading  Procedures.”  These  tasks  were  generally  centered  on  practical 
knowledge  and  skills  required  by  section,  fire  team,  and  squad  leaders  in  the  preparation, 
planning,  and  execution  of  successful  combat  operations.  From  this  list,  we  were  able  to  refine 
our  focus  to  a  narrower  skill  set. 

In  the  process  of  our  sample  selection,  we  reduced  the  pool  of  N  =  427  IMI  examples  to 
N=  179  by  focusing  on  IMI  relevant  to  MFE  and  to  new  Squad/Team  headers.  Using  the 
reduced  sample,  we  developed  a  scheme  to  classify  IMI  based  on  characteristics  related  to  its 
potential  reusability  in  a  point  of  need  context.  In  tandem  with  this  analysis,  we  reviewed  the 
existing  tailored  training  literature  to  identify  principles  that  could  be  applied  in  the  design  of 
IMI  to  improve  on  current  interactivity  and  tailoring  found  in  existing  IMI  courseware. 
Ultimately,  this  research  effort  was  exploratory  and  sought  to  point  the  way  ahead  for 


4  DLDC  was  formerly  known  as  Military  Interactive  Multimedia  Instruction  Center  (MIMIC). 
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implementing  point  of  need  concepts  within  current  and  future  IMI  repurposing  and 
development.5 

From  our  review,  it  was  clear  that  the  Anny  has  invested  extensively  in  IMI  and  other 
distributed  learning  (dL)  resources.  For  example,  prior  to  the  activation  of  the  MT2  site,  289 
IMI  courses  were  available  for  viewing  through  public  access  of  the  Reimer  Digital  Library 
(https://rdl.train.anny.mil/soldierPortal/).6  Courses  were  cataloged  with  each  course  or  domain 
assigned  a  specific  number.  For  example,  “Enter  a  Building  during  an  Urban  Operation”  was 
assigned  number  071-326-0513.  The  first  three  numbers  represent  the  proponent;  in  this  case, 
the  proponent  is  the  Infantry.  Many  existing  IMI  modules  are  identified  by  the  proponent- 
assigned  individual  or  collective  task  number.  Of  these  289  courses  in  the  Reimer  Digital 
Library,  196  were  categorized  under  the  Infantry  proponent  number  of  071;  three  are  categorized 
under  the  Field  Artillery  proponent  number  of  061 ;  two  are  categorized  under  the  Annor 
proponent  number  1 71.  The  remaining  courses  were  spread  throughout  the  Operations,  Support, 
and  Effects  and  Force  Sustainment  proponents.  Many  of  these  courses  were  migrated  to  the 
MT2  site  and  proponents,  such  as  the  FCoE,  have  made  access  available  to  additional  courses 
through  CoE-administered  sites.  Indeed,  the  Army  possesses  a  significant  quantity  of  IMI  and 
dL  materials.  However,  the  pedagogic  approaches  of  many  existing  IMI  tend  toward  a 
sequential,  book-like  structure.  This  type  of  structure  would  not  readily  facilitate  application  of 
the  ALM  point  of  need  learner-centric  techniques  because  of  its  linear  design  and  intended 
applicability  to  a  general  audience. 

The  MCoE  and  the  FCoE  provided  a  selection  of  current  IMI  materials  for  examination 
and  consideration.  This  selection  was  also  supplemented  by  a  review  of  IMI  posted  on  the  MT2 
site  that  was  relevant  to  MCoE.  A  summary  of  these  materials  is  in  Table  1.  A  more  complete 
listing  of  IMI  materials  reviewed  for  this  effort  is  provided  in  Appendix  A. 


5  In  a  subsequent  report,  we  will  present  IMI  exemplars  of  different  point  of  need  variations  as  well  as  the  tailored 
training  techniques  we  describe  here. 

6  This  hyper  link  has  been  redirected  to  the  MT2  site. 
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Table  1 

Summary  of  IMI  Courses  Reviewed  for  Potential  Reuse 


Course 

#  of  Lessons 

19K  (Armor  Crewman)  Advanced  Leader 

20 

19K  MOS  Transition 

30 

19D  (Cavalry  Scout)  Advanced  Leader 

21 

19D  MOS  Transition 

34 

13F  (Fire  Support  Specialist)  MOS 

28 

Stand-Alone  Lessons  (CD-ROM) 

Small  Unmanned  Ground  Vehicle  (SUGV)  Leaders 
SUGV  Employment  for  Leaders  (Platoon) 

Welcome  to  Jumpmaster 
Welcome  to  Pathfinder 

A  Day  in  the  Bam-  introduction  to  Virtual  Experience  Immersive  Learning  Simulation  (VEILS) 
Stand-Alone  Lessons  (MT2  Site) 

Engage  Targets  During  an  Urban  Operation 
Prepare  a  Range  Card  for  a  Machine  Gun 
Perform  a  Function  Check  on  an  MK19  Machine  Gun 
Unload  an  MK19  Machine  Gun 

Mount  a  Night  Vision  Sight,  AN/TVS-5,  on  an  MK19  Machine  Gun 

Prepare  Positions  for  Individual  and  Crew-Served  Weapons  During  an  Urban  Operation 

Enter  a  Building  During  an  Urban  Operation 

Correct  Malfunctions  on  an  MK19  Machine  Gun 
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Reviewing/Evaluating  IMI 


At  the  outset  of  the  review  process  we  determined  that  selected  courseware  would  be 
examined  in  terms  of  three  major  criteria:  (a)  relevant  and  current  subject  matter  content;  (b) 
design  features;  and,  (c)  if  the  courseware  was  compiled  in  a  manner  that  allowed  it  to  be 
separated  into  piece  parts  for  reuse.  This  third  point  is  a  significant  concern  if  point  of  need  is  to 
be  applied  using  existing  Anny  IMI.  Often,  separating  IMI  into  piece-parts  can  render  the 
individual  parts  inoperable,  undennining  a  reuse/repurposing  effort. 

Reviewers  included  operational  researchers,  military  subject  matter  experts,  educational 
and  psychological  researchers,  computer  programmers,  and  IMI  developers.  These  reviewers 
examined  the  IMI  courseware  from  their  individual  areas  of  expertise.  At  least  one  reviewer 
examined  each  piece  of  selected  IMI  courseware  in  terms  of  the  three  major  criteria.  In  cases 
when  a  single  reviewer  could  not  completely  evaluate  a  piece  of  IMI,  multiple  reviewers  were 
involved.  There  was  great  variability  in  time  (approximately  10  minutes  to  over  2  hours)  to 
complete  a  review  for  each  piece  of  IMI.  This  was  due  to  varying  sizes  of  IMI  courses;  that  is, 
the  number  of  lessons,  modules,  and  supplemental  material  that  had  to  be  evaluated. 

The  initial  step  was  to  examine  scientific  research  and  technical  references  to  determine 
viable  methods  for  evaluating  IMI  in  order  to  develop  an  effective  procedure  for  the  evaluation 
(e.g.,  Stoyanov  &  Kirschner,  2007).  The  evaluation  addressed  a  variety  of  related,  yet 
independent,  characteristics  of  IMI  concerning  their  potential  for  tailoring,  potential  for  reuse, 
and  potential  for  applications  of  learning  science  to  address  the  point  of  need  concept.  The  goals 
of  the  evaluation  were  to: 

•  Determine  a  strategy  for  categorizing  existing  IMI  that  would  support  identification  of 
materials  and  components  for  potential  tailoring  and  reuse. 

•  Determine  suitable  criteria  for  detennining  the  potential  for  IMI  tailoring  and  reuse. 

•  Identify  a  group  of  learning  principles,  instructional  techniques,  and  methods  suitable  to 
address  point  of  need  in  existing  and  future  IMI. 

The  initial  list  of  evaluation  criteria  was  developed  on  the  basis  of  similar  processes 
described  in  Stoyanov  and  Kirschner  (2007);  Kalyuga,  Chandler,  and  Sweller  (1999);  Metzler- 
Baddeley  and  Baddeley  (2009);  Bell  and  Kozlowski  (2002);  and  Eiriksdottir  and  Catrambone 
(2011).  From  these  sources,  we  derived  an  initial  list  of  criteria,  which  are  presented  in 
Appendix  B.  As  the  evaluation  process  was  iterative,  we  found  our  criteria  needed  to  be 
modified  as  we  progressed.  Many  of  the  criteria  we  had  initially  identified  were  not  applicable 
to  the  IMI  materials  we  evaluated.  As  we  proceeded,  the  evaluation  criteria  were  reduced.  Table 
2  presents  the  set  of  criteria  most  relevant  to  the  Army  IMI  we  evaluated. 
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Table  2 

Initial  Criteria  for  Evaluating  Existing  IMI  and  Associated  Point  of  Need  Considerations 


Criterion 

Description 

Point  of  Need  Design 

Considerations 

Complexity/Depth  of 
Information  Presented 

Is  there  sufficient  content  to  provide  the 
desired  coverage  of  the  subject  area, 
especially  when  attempting  to  form  the 
different  points  of  need  training  material? 

Can  aspects  of  the  information  present 
be  down-selected  to  focus  on  a 
specific  learning  need? 

Viable  Examples 

Flow  are  examples  being  used  and  are  there 
single  or  multiple  examples?  Multiple 
examples  may  be  better  to  allow  Soldiers  to 
generalize  principles  by  deriving  consistent 
patterns  across  examples  (cf.  Schwartz  & 
Bransford,  1998). 

Can  existing  examples  be  restructured 
or  reordered  to  apply  tailoring 
principles,  such  as  backward  fading? 

Are  the  examples  contextually 
relevant  to  the  target  audience? 

Narrative  Flow 

Does  the  narrative  of  the  training  make 
sense?  Do  instructional  pages  that  come  later 
logically  build  on  what  came  before? 

Would  additional  IMI  development  be 
required  to  produce  a  coherent 
narrative  when  restructuring  the 
existing  IMI? 

Presentation  is  Focused 
vs.  Diffuse 

Do  the  parts  have  a  clear  topical  focus,  or 
does  the  training  meander? 

Are  there  elements  of  the  existing  IMI 
that  can  be  removed  to  focus  the 
development  of  topics? 

Outcome  Meets  Goal 

Are  training  goals  achieved  by  the  end  of  the 
training?  Does  the  IMI  deliver  on  what  it 
promises  to  deliver? 

What  aspects  of  the  existing  IMI  need 
to  be  removed,  modified,  or  added  in 
order  to  address  the  identified  need  of 
a  specific  audience? 

Grouping  of  Content 

Are  the  modules  and  information  grouped  in 
a  way  that  makes  sense  and  provides  a 
coherent  structure  (i.e.,  support  development 
of  schemas)? 

Is  the  essential  information  present  to 
address  the  depth  of  knowledge 
required  by  the  specific  audience  to 
accomplish  the  task  being  trained? 

Timing 

Would  a  Soldier  be  able  to  develop 
understanding  of  one  learning  point  before 
the  next  one  is  presented?  Are  there  logical 
points  at  which  a  Soldier  can  take  a  break 
from  what  they  are  learning? 

Is  it  possible  for  a  learner  to  jump 
around  in  the  IMI  to  focus  only  on 
topics  that  are  relevant  to  them  at  a 
specific  point  in  time? 

After  identifying  the  most  salient  criteria  for  evaluation,  we  categorized  IMI  materials 
and  components  for  modification  to  a  point  of  need  format.  This  process  was  supported  by  some 
research  team  members’  previous  experiences  developing  multimedia  courseware.  The  team 
examined  characteristics  of  existing  IMI  that  would  contribute  to  a  meaningful  IMI 
categorization.  The  goal  was  to  identify  criteria  that  assisted  in  selecting  IMI  for  reuse  as 
familiarization,  core,  or  tailored  training  IMI. 
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The  process  initially  identified  multiple  characteristics  of  IMI.  These  characteristics 
consisted  of  administrative  details,  such  as  title  and  time;  pedagogical  information  such  as 
training  taxonomy,  tenninal,  and  enabling  learning  objectives;  and  technical  characteristics  that 
included  visual  media  and  text  file  details. 

Although  the  initial  goal  was  to  examine  IMI  to  identify  modules  with  characteristics 
favorable  to  modification  to  a  point  of  need  fonnat,  many  of  our  expectations  were  not  met  that 
would  allow  us  to  select  from  a  large  pool  of  IMI  for  reuse/modification.  This  limitation 
emerged  from  our  concerns  that  the  IMI  must  be  able  to  meet  the  needs  of  a  predetermined,  very 
specific  audience.  In  our  case,  we  were  focusing  on  new  squad/team  leaders  (i.e.,  soon  to  be 
promoted  Specialists/Corporals  and  Sergeants).  We  needed  to  take  a  step  back  in  our  evaluation 
process  to  look  at  more  fundamental  issues  guiding  selection.  Many  IMI  products  reviewed  did 
not  consistently  contain  the  depth  of  infonnation,  repurposable  graphics  and  images,  viable 
examples  in  the  instructional  material,  and/or  appropriate  tests  to  appropriately  target  a  specific 
audience.  Rather  than  work  from  a  more  complex  typology  based  on  many  criteria  (i.e.,  those 
described  in  Table  2),  we  decided  to  initially  focus  only  on  those  factors  that  could  most 
practically  be  used  to  guide  selection  and  then  move  toward  the  more  complex  evaluation  as  we 
were  able  to  down-select  our  sample  of  IMI. 

The  final  selection  criteria  focused  on  broader  characteristics  that  would  more  readily 
indicate  the  potential  of  particular  IMI  for  modification  to  a  point  of  need  training  fonnat.  These 
characteristics  were: 

•  Source  file  availability, 

•  Doctrinal  accuracy  and  currency  of  content,  and 

•  Chunks  of  content  that  could  be  used  to  satisfy  the  point  of  need  requirement,  with 
respect  to  the  criteria  described  in  Table  2. 

This  simplified  approach  is  depicted  in  Figure  1.  Rather  than  filtering  through  myriad  criteria,  it 
was  apparent  that  a  few  central  characteristics  were  sufficient  to  identify  IMI  that  had  potential 
to  be  modified  to  a  point  of  need  fonnat,  which  we  could  then  evaluate  in  tenns  of  the  criteria 
described  in  Table  2. 
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Typology  for  Selecting  IMI  for  Modes  of 

Adaptive/Tailored  Training 


Potential 

Reusable 

Materials 


FILTER  CRITERIA 


Source  File 
Availability 


Doctrinal  Accuracy 
and  Currency 


Content  chunks  that 
meet  point  of  need 
requirement 


Available  to  Tailor  for 
Point  of  Need  Training 


Are  source  files  available? 
Are  the  source  files 
searchable  and  extractable? 

Are  methods  and  concepts 
used  in  the  materials 
current? 

Are  uniforms,  equipment, 
and  materials  depicted 
current? 

Are  pieces  or  chunks 
available  (both  instructional 
content  and  test  materials) 
suitable  to  support  tailored 
training  approaches  for 
point  of  need  training? 


Figure  1.  Typology  for  Selecting  IMI  for  Modes  of  Adaptive/Tailored  Training. 

Observations  Emerging  from  the  Evaluation 
Structural  Characteristics  of  Reviewed  IMI 

Our  initial  review  of  the  IMI  products  indicated  a  wide  variety  of  pedagogical  approaches 
and  media  uses.  However,  most  IMI  appeared  to  be  structured  for  familiarization  or  as  an 
introduction  to  knowledge,  tasks,  or  skills.  Information,  knowledge,  and  skills  were  seldom 
presented  in  the  context  of  more  than  a  single  situation.  Checks  on  learning  or  module  and 
section  tests  closely  followed  the  presentation  of  information.  The  same  questions  used  as 
knowledge  checks  were  often  used  throughout  the  lessons  and  appeared  as  final  assessment 
questions.  Seldom  were  opportunities  provided  to  permit  the  application  or  practice  of 
information  in  the  context  of  a  whole  task. 

The  ALM  concept  of  using  existing  IMI  for  “just  in  time”  and  “as  needed,  when  needed” 
may  be  unattainable  through  existing  courseware  when  the  focus  is  on  tailoring  to  the  differing 
needs  of  particular  individuals.  For  example,  the  13F  Fire  Support  Specialist  site  available 
through  the  FCoE  currently  contains  over  80  hours  of  quality,  doctrinally  correct,  training 
materials  accessible  to  an  ambitious  student  of  the  art  of  fire  support.  However,  the  more 
experienced  13F  specialist,  Infantry  squad  leader,  tank  commander,  or  reconnaissance  section 
sergeant  looking  for  a  quick  (i.e.,  a  few  minutes)  refresher  of  core  knowledge  concerning  call- 
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for- (Ire  skills  or  desiring  to  familiarize  himself  in  a  short  period  of  time  with  mortar  registration 
requirements,  would  be  frustrated  with  the  difficulty  of  locating  this  content  in  the  existing  IMI 
courseware. 

Some  of  the  courseware  provided  by  the  FCoE  proved  to  be  an  exception  to  this 
generalization  due  to  the  organization  of  the  materials.  The  lessons  within  these  courses  were 
typically  organized  into  five  distinct  parts  or  elements: 

•  Introduction  -  provided  administrative  information  and  the  organization  of  the  material. 

•  Tutorial  -  provided  primary  training  on  how  to  perform  the  task  and  conveyed  the  needed 
knowledge,  tasks,  and  skills. 

•  Demo  -  demonstrated  an  expert  performing  the  task  from  start  to  finish. 

•  Guided  Practice  -  guided  the  student  through  the  complete  perfomiance  of  the  task. 

•  Knowledge  Review  -  provided  a  review  of  the  key  performance  measures  in  the  lesson  and 
provided  a  check  on  learning  or  test  for  assessment. 

This  crawl-walk-run  approach  to  training  contrasted  with  the  three-frames  and  a  check  on 
learning  approach  used  in  a  number  of  other  IMI  lessons.  Additionally,  some  course  materials 
used  role  playing  or  followed  the  role  of  a  character  in  a  situation.  These  products  engaged  the 
learner  using  video  or  animation,  providing  an  environment  that  comes  close  to  being  immersive. 
Even  so,  a  distinction  needs  to  be  made  between  immersive  training  and  tailored  training.  The 
learner  selected  choices  that  drive  the  situation  and  prompted  the  student  to  “think  again”  or 
reselect  when  their  choice  proved  wrong.  The  new  choice  prompted  a  replay  of  the  situation 
with  varied  severity  of  consequences  and  results;  better  decisions  mitigated  the  more  negative 
consequences  or  proved  successful.  Immersive  training  gives  the  user  some  control  of  the 
emerging  storyline  the  training  follows;  however,  this  control  does  not  give  the  learner  control 
over  selecting  training  to  address  his  or  her  preferences  and  needs. 

Most  products  included  knowledge  check  questions.  Questions  incorporated  correct  and 
incorrect  feedback.  In  general,  the  approach  following  the  presentation  of  incorrect  feedback 
was  as  follows.  With  the  first  incorrect  feedback  the  learner  received  a  prompt  or  hint.  The 
question  was  presented  a  second  time,  sometimes  the  response  options  were  randomized,  and  in 
others  they  remained  the  same.  If  the  learner,  incorrectly  responded  a  second  time,  the  directions 
told  the  student  to  review  the  content  again.  This  rerouted  the  learner  to  the  topic  beginning. 
When  the  learner  reviewed  previously  viewed  content,  the  same  question  was  again  presented. 
Questions  could  be  designed  to  better  target  the  learners’  specific  areas  of  difficulty  in  learning 
the  material,  with  less  repetition. 

Many  courses  followed  a  linear  sequence  even  with  incorporated  animations  and  question 
sequences.  Learners  had  to  follow  the  prescribed  lesson  path  until  the  lesson  was  completed. 
There  were  no  options  that  allowed  for  individual  learning  differences  and  preferences.  The 
content  and  structure  of  most  IMI  courseware  appeared  focused  on  a  particular  structure  of 
presentation  for  topics,  skills,  or  tasks,  a  template  that  was  frequently  repeated  across  different 
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IMI.  The  design,  structure,  and  sequence  guided  the  learner  along  a  single  path  of  learning  to 
completion  without  the  Soldier  being  required  to  do  much  except  select  “NEXT”  and  recall 
terms  or  answer  simple  questions  that  echoed  key  points  presented  in  the  training.  Some  courses 
had  animation.  However,  most  had  little  Soldier  interaction  beyond  answer  selection,  “NEXT” 
navigation,  and  pop-up  boxes. 

While  a  few  courses  allowed  a  user  to  navigate  to  selected  sections  or  topics  within  the 
courseware,  their  primary  organization  and  structure  facilitated  taking  the  course  from  start  to 
finish,  rather  than  focusing  on  isolated  sections  or  accessing  training  for  only  selected  knowledge 
or  skills.  Few  courses  were  designed  to  permit  rapid  navigation  to  isolated  skills,  part-tasks, 
specific  knowledge  domains,  or  for  a  specific  point  of  need.  Menus  and  course  maps  seldom 
identified  with  clarity  the  task,  skill,  or  knowledge  contained  in  a  lesson  or  module.  Menus  were 
not  consistent  throughout  the  courses.  The  specific  skill  or  small  kernel  of  knowledge  that  the 
Soldier  may  require  was  sandwiched  in  the  middle  of  a  course  or  distributed  throughout  multiple 
lessons. 

For  learners  to  acquire  or  be  exposed  to  their  needed  knowledge  or  skills,  the  IMI 
reviewed  would  require  the  learner  to  complete  the  entire  course  or  to  access  multiple  lessons  or 
modules  within  a  course.  Few  courses  were  organized  to  provide  a  quick  familiarization  with 
learner-detennined  subject  or  a  track  to  gain  core  knowledge  of  domain-specific  tasks.  At  times, 
the  desired  skill  or  knowledge  was  camouflaged  within  the  courseware;  present  in  the  menu,  but 
hidden  from  easy  access  by  learners.  In  many  cases,  the  learners  would  have  needed  to  work 
through  the  entire  course  even  if  they  had  already  mastered  the  necessary  knowledge  or  skills 
contained  in  sections  of  the  courseware.  Proficiency  or  test-out  options  were  not  generally 
available. 

Addressing  the  IMI  Reuse  Question 

When  we  initially  planned  this  research  effort,  we  had  expected  that  reuse  of  existing  IMI 
would  expedite  our  ability  to  execute  modifications  to  address  point  of  need.  We  found  the  reuse 
process  far  more  complicated  than  initially  anticipated.  The  issues  we  encountered  are  widely 
recognized  in  the  research  literature  (for  review  see  ‘The  SCORM  Debate’  in  VET  Learning 
Object  Repository,  2003).  Reuse  of  IMI  courseware  components  is  based  on  the  assumption  that 
whole  lessons,  modules,  or  SCOs  can  be  broken  apart  and  reassembled  in  a  plug  and  play 
fashion.  Even  if  this  was  possible,  this  approach  would  not  address  how  content  within  a 
repurposed  IMI  module  could  be  selected  and  reshaped  to  better  address  a  learner’s  specific 
needs.  Given  the  difficulties  we  encountered  on  a  small  scale,  we  believe  that  any  large  scale 
reuse/repurposing  of  existing  IMI  programming,  graphics,  etc.,  to  address  point  of  need  may  be 
time  and  cost  prohibitive.7 

Adapting  IMI  to  Point  of  Need 

Setting  aside  questions  of  reuse/repurposing  of  IMI,  we  considered  ways  to  implement 
point  of  need  within  IMI  more  generally.  A  major  consideration  when  examining  the  existing 

7  This  issue  will  be  addressed  in  greater  detail  in  a  subsequent  research  report. 
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IMI  was  to  determine  what  would  be  required  to  incorporate  various  tailoring  principles  to 
address  the  point  of  need  with  an  eye  toward  future  development  of  IMI. 

Dyer,  Wampler,  and  Blairkenbeckler  (2011)  identified  forms  of  tailored  training, 
provided  examples  of  tailored  training  in  Anny  courses,  and  even  offered  explanations  why 
selected  forms  of  tailoring  might  be  more  applicable  to  Army  training.  In  our  research,  we 
considered  a  mix  of  approaches  to  tailoring  IMI.  Of  the  approaches  considered,  all  focused  on 
macro-level  adaptations,  since  there  often  would  not  be  an  instructor  present  when  Soldiers  are 
completing  IMI  modules.  These  approaches  are  described  in  Table  3. 

Table  3 


Macro-Level  Adaptations  to  IMI  to  Address  Point  of  Need 


Approach 

Description 

Preliminary,  whole-task  evaluation 

At  the  outset  of  training,  Soldiers  are  asked  to  complete  a  scenario-based 
evaluation  over  a  whole  task  relevant  to  the  domain  to  be  trained 

Provide  specific,  detailed  feedback 

Following  the  whole-task  evaluation,  Soldiers  are  provided  specific  Go/No 

Go  feedback  on  each  of  the  subtasks  covered  by  the  whole-task  test. 

Allow  Soldier  to  choose  among 
various  training  options  adapted  to 
points  of  need 

Given  feedback  on  their  performance,  Soldiers  are  then  given  the  option  to 
pursue  additional  training  to  help  them  (a)  better  understand  concepts 
relevant  to  the  domain,  (b)  execute  the  task,  (c)  go  through  a  worked 
example  with  scaffolding  and  ongoing  feedback  on  their  performance. 

Post-training  whole-task  evaluation, 
again  with  specific,  detailed  feedback 

Provide  the  Soldier  with  an  opportunity  to  evaluate  how  well  they  learned 
the  critical  skills  taught  in  the  IMI.  Here,  Soldiers  are  again  presented 
options  to  pursue  additional  training,  with  each  option  targeted  to  their 
point  of  need. 

Currently,  Army  institutional  training  is  not  usually  tailored  to  student  needs,  similar  to 
what  is  seen  with  most  existing  IMI  (Dyer,  Wampler,  &  Blairkenbeckler,  2011).  In  the 
classroom,  instructors  may  lack  experience  in  identifying  common  difficulties  students  encounter 
when  learning  particular  tasks  and  skills.  Moreover,  instructor  preparation  courses  rarely  cover 
techniques  and  tools  for  tailoring  course  content  to  students’  needs.  Course  instruction  often  is 
given  directly  from  standard  lesson  plans,  with  no  adjustment  for  the  training  audience.  Training 
techniques  are  seldom  altered  to  the  task  and/or  the  knowledge  and  experience  of  the  training 
audience  (cf.  Bickley  et  ah,  2010;  Dyer  et  ah,  2000;  Leibrecht,  Wampler,  Goodwin,  &  Dyer, 
2007;  Tucker,  McGilvray,  &  Leibrecht,  2009;  Wampler,  Dyer,  Livingston,  Blairkenbeckler,  & 
Dlubac,  2006;  Wampler,  James,  Leibrecht,  &  Beal,  2007).  While  standardized  lessons  are  the 
norm  for  Army  institutional  training,  the  IMI  format  presents  an  opportunity  for  tailoring,  since 
IMI  lessons  are  most  often  completed  individually.  Tailoring  techniques  that  may  be  infeasible 
to  apply  in  a  classroom  setting  may  in  fact  be  very  applicable  in  an  IMI  learning  context. 

For  the  ALM  to  be  successful,  the  Army  should  look  for  opportunities  to  apply  tailored 
training  techniques  across  a  broad  spectrum  of  training  environments,  modalities,  and  materials. 
Tailoring  training  should  accommodate  student  preferences  for  learning  while  also  maintaining 
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sound  learning  strategies.  One  resource  to  inform  the  tailoring  process  is  the  First  Principles  of 
Instruction  developed  by  Merrill  (2002).  The  principles  fonn  an  instructional  outline  created 
from  examining  numerous  instructional  models  and  theories.  Merrill’s  approach  has  found 
support  in  other  instructional  design  research  (see  Van  Merrienboer  &  Sweller,  2005;  Sweller, 
Kirschner,  &  Clark,  2007). 

Merrill  (2002)  detennined  a  small  set  of  principles  of  instruction  are  present  across  good 
instruction,  regardless  of  type.  He  describes  these  principles  as  interrelated.  Further,  when  these 
principles  are  implemented  in  an  instructional  program,  they  resulted  in  increased  quality  of 
instruction  and  amount  of  learning.  According  to  Merrill,  instruction  needs  to  (a)  be  task 
(problem)-centered,  (b)  activate  prior  knowledge  and  experience,  (c)  include  demonstrations  of 
tasks  and  skills,  (d)  require  learners  to  apply  what  they  have  learned,  and  (e)  encourage  learners 
to  integrate  their  new  knowledge  into  their  everyday  world.  Table  4  provides  a  descriptive 
summary  of  each  of  Merrill’s  principles. 


Table  4 

Merrill’s  (2002)  First  Principles  of  Instruction 


Principle 

Description 

Tailored  Training  Application 

Task  (Problem)- 
C entered 

Learning  is  promoted  when  learners  engage 
in  a  task-centered  instructional  strategy. 

The  task  should  be  a  problem  that  the 
student  may  encounter  in  a  real-world 
situation.  This  principle  lies  at  the  center  of 
the  others. 

Whole-task  focused  assessment 
with  detailed  feedback  to  learner 
about  performance  on  particular 
aspects  of  the  task. 

Activation 

Learning  is  promoted  when  learners  activate 
prior  knowledge  or  experience.  Reminding 
the  learner  of  relevant  previous  experience 
promotes  learning  by  allowing  them  to 
build  upon  what  they  already  know.  It  may 
also  be  important  to  provide  experience 
when  a  learner’s  previous  experience  is 
inadequate.  This  helps  to  develop  useful 
mental  models  that  are  similar  in  structure 
to  the  content  being  taught,  and  that  help  to 
integrate  new  information. 

Various  scaffolding  techniques 
based  on  testing  and/or  reported 
experiences.  Presenting  templates, 
mental  models,  and  other 
cognitively  organizing  structures 
throughout  the  training. 

Demonstration 

Learning  is  promoted  when  learners  observe 
a  demonstration  of  the  task  or  skill.  Most 
effective  are  simulations,  visualizations,  and 
modeling  that  exemplify  what  is  being 
taught.  Demonstration  may  include  guiding 
learners  through  different  representations  of 
the  same  experience  through  the  use  of  a 
media,  pointing  out  variations,  and 
providing  key  infomiation. 

Present  series  of  examples  and  help 
the  learner  to  derive  a  generalized 
mental  model.  Use  ongoing  testing, 
with  feedback,  to  help  shape 
learners’ mental  models.  Target 
training  to  learners’  specific  areas 
of  difficulty. 
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Table  4  (continued) 

Merrill’s  (2002)  First  Principles  of  Instruction _ 

Principle _ Description _ 

Application  Learning  is  promoted  when  learners  apply 

the  new  knowledge.  This  principle  requires 
that  learners  use  their  new  knowledge  in  a 
problem-solving  task,  using  multiple  yet 
distinctive  types  of  practice.  The 
application  phase  should  be  accompanied 
by  feedback  and  guidance  that  is  gradually 
withdrawn  as  the  learners'  capacities 
increase  and  performance  improves. 


Tailored  Training  Application 

Backwards  fading.  As  a  learner 
begins  to  become  proficient  with  an 
aspect  of  a  task,  remove  the 
instructional  supports  for  that  aspect 
of  the  task. 


Integration 


Learning  is  promoted  when  learners 
integrate  their  new  knowledge  into  their 
everyday  world  (mission  or  job 
environment).  Effective  instruction  occurs 
when  learners  can  demonstrate,  adapt, 
modify  and  transform  new  knowledge  to 
suit  the  needs  of  new  situations.  Learners 
may  also  teach  others.  Information  sharing 
is  important  for  new  knowledge  to  become 
part  of  a  learner's  personal  repertory  and  for 
learners’  sense  of  progress.  Collaborative 
work  in  a  group  can  provide  a  context  for 
this  phase. 


Present  learners  with  familiar  and 
novel  examples,  problems,  and 
applications  based  on  their  indicated 
background  experiences  and/or 
testing.  Exposure  to  training 
material  and  application  problems 
that  contain  both  familiar  and  novel 
elements  tailored  to  learners’  needs 
may  help  increase  longer-term 
transfer. 


The  principles  described  above  could  serve  as  the  foundation  for  guiding  the  tailoring  of 
training  during  implementation  of  the  ALM  more  generally,  as  well  as  in  IMI  applications.  To 
address  the  first  principle,  IMI  could  incorporate  multiple,  complex  examples  of  real-world 
events  and  scenarios  relevant  to  the  knowledge  and  skills  the  IMI  addresses.  Second,  references 
to  previous  learning — perhaps  indicating  the  particular  level  of  knowledge  and  skills  an  IMI  is 
targeting — could  be  incorporated.  In  addition,  templates  and  mental  models  could  be  presented 
to  assist  the  learner  in  structuring  the  new  information  they  will  be  learning  in  the  context  of 
knowledge  and  experiences  they  have  already  had.  Third,  demonstrations  and  compelling 
examples  should  be  used  throughout  the  IMI  training.  When  learners  are  exposed  to  multiple 
contrasting  cases/examples,  they  are  better  able  to  derive  generalizable  principles  and  establish  a 
mental  model  for  what  they  are  learning  (Schwartz  &  Bransford,  1998).  Fourth,  as  learners 
progress  through  the  training,  they  should  be  encouraged  to  apply  what  they  have  learned  to 
address  problem-solving  scenarios;  with  scenarios  presented  later  in  training  providing  fewer 
contextual  cues  and  learning  supports  than  those  presented  earlier  in  training.  Finally,  the  IMI 
should  focus  on  concrete  examples  of  how  the  learned  knowledge  and  skills  can  be  integrated 
into  new  contexts  and  situations.  It  may  be  helpful  to  include  at  least  one  novel  or  unexpected 
application  of  learned  knowledge  and  skills  toward  the  end  of  the  training  to  get  the  learner 
thinking  in  terms  of  longer  term  transfer  of  what  they  have  learned. 
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As  stated  previously,  the  IMI  lessons  we  reviewed  were  generally  broad  in  scope  and  did 
not  address  specific  points  of  need.  While  lessons  and  even  modules  contain  useful  infonnation 
and  material,  the  content  was  generally  designed  and  developed  as  one-size-fits-all.  That  is,  the 
IMI  was  intended  to  fill  several  purposes,  and  was  designed  to  be  generally  applicable — not  to 
target  specific  individual  needs.  Merrill’s  First  Principles  (2002)  provide  an  ideal  starting  point 
for  repurposing  or  developing  IMI  for  specific  points  of  need.  In  addition,  to  receive  high 
student  satisfaction,  the  point  of  need  IMI  should:  (a)  link  knowledge  and  skills  to  the  context  of 
job  and  mission  environments  through  examples  and  applications,  (b)  encourage  learners  to 
practice  new  knowledge  and  skills,  (c)  provide  learning  content  in  chunks  to  make  locating 
specific  information  easier,  and  (d)  engage  students  in  a  meaningful  learning  experience  through 
demonstrations  and  integration  of  knowledge  and  skills  with  their  everyday  lives. 

Discussion 

This  was  an  exploratory  research  effort  focusing  on  applications  of  tailored  training 
learning  strategies  and  design  techniques  to  incorporate  the  ALM’s  point  of  need  concept  into 
Army  IMI.  We  reviewed  a  sample  of  Army  IMI  from  the  Maneuver,  Fires,  and  Effects  domain, 
relevant  to  the  specific  needs  of  new  Squad/Team  Leaders.  In  this  research,  we  detennined  a 
number  of  relevant  learning  principles  and  design  techniques  that  could  be  employed  to  support 
implementation  of  point  of  need  in  Army  IMI,  although  we  also  found  that  to  implement  such 
principles  and  techniques  using  existing  Army  IMI  may  be  prohibitive  on  a  large  scale  due  to 
time  and  cost.  Point  of  need  represents  a  fairly  significant  divergence  from  the  design 
philosophies  guiding  the  development  of  earlier  Army  IMI,  moving  as  it  does  from  an  emphasis 
on  general  applicability  to  an  emphasis  on  the  unique  particulars  of  individuals  and  their  learning 
situations. 

Summary  of  Findings  and  Recommendations 

After  reviewing  what  has  been  detennined  from  others’  efforts  to  reuse  IMI,  we  were  not 
optimistic  that  our  endeavor  would  yield  a  different  outcome.  Based  on  our  evaluation,  most  of 
existing  Army  IMI  likely  could  not  be  repurposed  for  point  of  need  without  significant 
investment  in  resources  and  time.  However,  a  number  of  tailored  training  learning  principles 
could  be  applied  in  future  developments  of  IMI  to  address  point  of  need  challenges  and  create 
the  flexibility  needed  for  future  repurposing  or  modifications.  On  the  basis  of  our  evaluation  and 
review  of  a  selection  of  Army  IMI  for  application  of  the  ALM  point  of  need  concept,  we 
developed  a  number  of  recommendations. 

Future  point  of  need  IMI  should  be  designed  for  a  well-defined  audience  and  targeted  to 
individual  needs.  A  design  philosophy  that  focuses  on  developing  generally  applicable,  one- 
size-fits-all  IMI  is  less  applicable  in  the  context  of  ALM.  Shifting  to  a  point  of  need  design 
philosophy  encourages  the  Anny  to  address  both  issues  of  accessibility  and  the  capability  of  the 
learning  resource  to  be  tailored  to  individual  learners’  needs. 
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Point  of  need  seems  to  be  a  matter  of  striking  a  balance  between  predetermining  the 
structure  of  the  IMI  content  and  allowing  learners  to  make  self-aware  choices  about  their 
learning  experience.  On  this  basis,  specific  design  features  of  stand-alone  IMI  to  address  point 
of  need  may  include: 

•  providing  well-structured  IMI,  with  high  learner  support,  and  multiple  user-selectable 
learning  paths; 

•  linking  knowledge  and  skills  to  the  job  contexts  and  mission  environments  through 
examples  and  applications; 

•  encouraging  learners  to  practice  new  knowledge  and  skills  by  focusing  on  developing 
mental  models  and  facilitating  far  transfer; 

•  providing  learning  content  in  smaller,  self-contained  chunks  (a)  to  make  locating 
particular  knowledge  and  skills  an  easier  task,  (b)  to  facilitate  training  within  the  duty 
day,  and  (c)  to  help  users  form  stronger  conceptual  models  of  the  training  domain; 

•  engaging  students  in  a  meaningful  learning  experience  through  demonstrations  and 
integration  of  knowledge  and  skills  with  their  everyday  lives; 

•  using  whole-task  pre-  and  post-assessments  to  help  learners  develop  higher  self- 
awareness  of  their  level  of  knowledge  and  skills  with  respect  to  the  domain; 

•  allowing  learners  greater  autonomy  in  choosing  how  they  proceed  through  their  learning 
experience — i.e.,  provide  learning  paths  designed  for  specific  points  of  need;  and 

•  recognizing  that  good  pedagogy  and  good  content  are  likely  more  important  than  costly 
high  bandwidth  multimedia  in  capturing  and  maintaining  learners’  interest  and 
motivation. 

Even  though  IMI  should  be  developed  for  specific  needs,  designers  should  keep  potential 
reuse/repurposing  in  mind.  Learning  modules  and  chunks  need  to  be  built  for  stand-alone  use. 
The  component  parts  (e.g.,  graphic  images,  narrations,  text  files)  should  be  constructed  and 
labeled  for  easy  identification  and  potential  changes.  Connections  to  an  LMS  should  be  minimal 
and  a  means  to  locate  and  alter  these  hooks/ties  should  be  documented. 

Learning  design  structures.  Learning  design  structure  concerns  how  the  overall 
progress  of  topics  and  supporting  infonnation  is  presented  in  the  IMI.  In  general,  we 
recommend  techniques  that  support  well-structured  learning  designs,  rather  than  open-ended 
designs.  Well- structured  learning  designs  have  been  found  to  be  more  effective  and  efficient 
than  exploratory  designs,  which  offer  little  to  no  guidance  to  students  (Kirschner,  Sweller,  & 
Clark,  2006).  On  this  basis,  some  techniques  can  be  readily  incorporated  into  IMI  to  reduce 
cognitive  load  and  support  longer-term  transfer  and  problem-solving  skills.  For  instance, 
learners  who  practiced  skills  with  guided  support  (i.e.,  using  worked  examples  or  process 
worksheets)  tended  to  outperform  learners  who  were  left  on  their  own  to  discover  appropriate 
procedures.  In  fact,  Kirschner  and  colleagues  found  that  experienced  learners  tended  to  gravitate 
to  courses  that  provided  more  learning  support  and  guidance,  a  finding  also  supported  by  Straus 
et  al.  (2009),  Dyer  and  Salter  (2001),  and  Dyer,  Singh,  and  Clark,  2005.  In  particular,  Dyer, 
Singh,  and  Clark  found  that  Soldiers  in  One  Station  Unit  Training  benefited  from  more 
structured  computer-based  training  whereas  trainees  in  the  Infantry  Officer  Basic  Course 
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preferred  and  were  better  prepared  through  computer-based  training  that  allowed  them  to  select 
among  various  content  and  training  approaches.  Dyer  and  Salter  found  that  chunking 
information  into  smaller  segments  allowed  Soldiers  to  better  retain  what  they  had  learned  as 
compared  to  an  open-ended,  exploratory  training  fonnat. 

However,  this  does  not  mean  that  more  open-ended  designs — with  lower  levels  of 
support  and  guidance — are  without  application  in  future  IMI  development.  Discovery  learning  is 
most  effective  when  learners  have  prior  experience  in  a  domain.  They  have  already  developed 
an  understanding  of  how  things  work  within  that  domain,  an  understanding  that  then  can  be 
applied  to  make  sense  of  the  open-ended  problems  they  are  exposed  to  in  the  discovery  learning 
fonnat  (cf.  Dean  &  Kuhn,  2006).  Optimally,  the  place  for  open-ended  designs  would  be  in 
blended  learning  situations  in  which  the  IMI  is  being  used  to  support  collaborative  learning 
activities  among  groups  of  learners  who  are  supported  by  a  facilitator. 

Optimally,  it  seems,  Army  IMI  should  at  the  appropriate  times  incorporate  both  types  of 
design  elements  to  establish  knowledge  structure  (such  as  direct  instruction)  and  elements  to 
explore  and  modify  preexisting  knowledge  structures  (such  as  discovery  learning,  problem-based 
learning,  and  other  constructivist  approaches).  Therefore,  when  tailoring  IMI  for  reuse  or 
developing  new  IMI,  merely  offering  total  flexibility  for  a  student  to  self-detennine  how  to 
proceed  would  not  be  expected  to  be  as  effective  as  providing  a  tailored  approach  that 
incorporates  sufficient  structure  to  support  the  learning  process,  yet  also  gives  the  student  an 
ability  to  make  infonned  choices  about  his  or  her  learning  experience.  We  recommend  using 
scenario-based  whole-task  evaluation  followed  with  detailed  feedback  and  various  learning 
paths — addressing  different  points  of  need — in  order  to  facilitate  this  type  of  design  structure. 
With  this  type  of  design,  students  receive  both  well-structured  instruction  and  learning 
experiences  that  can  be  tailored  to  their  specific  needs. 

In  addition,  it  should  be  noted  that  no  matter  how  well-designed  an  IMI  application  is,  if 
students  lack  motivation,  their  performance  will  suffer.  Exposing  students  to  whole-task 
problems  before  and  during  instruction — supplementing  instruction  with  part-task  practice  and 
supportive  information — may  serve  to  motivate  students  by  helping  them  recognize  and  want  to 
address  their  own  knowledge  and  skill  deficiencies  (Van  Merrienboer  &  Kester,  2007;  Van 
Merrienboer  &  Kirschner,  2013). 

The  design  elements  discussed  above  can  be  incorporated  into  IMI  to  support 
implementation  of  ALM.  In  particular,  IMI  can  (a)  be  designed  to  apply  both  highly-structured 
and  more  discovery-oriented  learning  approaches  depending  on  the  intended  context  of  use,  (b) 
intersperse  whole-task  and  part-task  problem  solving  activities  to  motivate  learners,  (c)  use  task- 
specific  feedback  to  help  learners  develop  awareness  of  their  learning  needs,  and  (d)  include 
choices  among  different  learning  paths  to  give  learners  autonomy  and  control  over  their  learning 
process,  enabling  them  to  address  specific  points  of  need.  With  these  design  elements,  IMI 
becomes  more  facilitative  and  tailored  to  individual  needs,  without  incurring  steep  increases  in 
development  costs  due  to  incorporation  of  bandwidth-heavy  multimedia  features. 
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Tailored  Training  Techniques.  The  described  approach  to  designing  IMI  for  point  of 
need  incorporates  a  variety  of  longstanding  tailored  training  techniques.  These  include  ability 
grouping,  self-pacing  of  instruction,  and  aptitude  by  treatment  interactions. 

Ability  grouping.  In  a  classroom  setting,  ability  grouping  refers  to  the  practice  of  putting 
students  into  groups  on  the  basis  of  individual  group  members’  ability  levels.  At  times,  members 
of  a  group  may  all  be  of  a  similar  level  of  ability;  other  times,  students  who  have  lower  ability 
may  be  grouped  with  a  student(s)  of  higher  ability.  After  students  have  been  grouped, 
instruction  is  tailored  to  address  the  particular  needs  of  the  group.  A  similar  situation  may  be 
addressed  in  IMI  by  using  pretesting  to  determine  a  student’s  level  of  ability  and  then  providing 
training  to  them  that  is  designed  meet  needs  associated  with  their  particular  level  of  knowledge 
and/or  skills.  A  posttest  with  feedback  could  be  included  with  each  tailored  IMI  module.  The 
posttest  is  intended  to  allow  the  student  to  assess  his  or  her  knowledge  after  completing  the 
instruction.  This  allows  the  student  to  make  an  informed  decision  as  to  whether  he  has  learned 
the  necessary  material,  should  repeat  any  of  the  blocks  of  instruction,  or  should  consider 
augmenting  his  or  her  understanding  by  engaging  more  in-depth  learning  materials. 

Self-paced  instruction.  Self-pacing  allow  students  to  adjust  their  learning  progress  to 
their  own  speed.  While  most  IMI  modules  pennit  students  to  control  their  speed,  we  recommend 
designing  IMI  to  consist  of  multiple  blocks  that  vary  in  depth  of  instruction.  This  design  feature 
may  offer  students  increased  flexibility  in  how  rapidly  they  progress  through  the  material.  In 
combination  with  a  pretest  and  detailed  perfonnance  feedback,  students  would  be  able  to  choose 
their  own  sequence  for  completing  the  blocks  of  training. 

Prior  knowledge.  Another  approach  to  tailoring  IMI  is  to  use  aptitude-by-treatment 
interactions,  a  design  feature  in  which  the  instructional  material  is  focused  on  the  aptitudes  of  the 
learners  (Snow,  1989).  Students  with  experience  in  the  IMI  module  content,  i.e.,  high-prior- 
knowledge  individuals,  can  select  a  training  path  with  lower-structure  instruction,  such  as  those 
activities  focused  on  solving  a  problem.  Students  with  less  experience,  i.e.,  lower-prior- 
knowledge  individuals,  can  select  a  learning  path  with  more  highly-structured  instruction, 
incorporating  features  such  as  step-by-step  instruction  and  practice  exercises  with  feedback  or 
worked  examples. 

The  Future  of  Army  IMI 

Straus  et  al.  (2011)  described  what  the  future  of  Anny  IMI  may  look  like.  They  note  that 
the  Army  Distance  Learning  Program  (TADLP)  funding  for  developing  content  has  declined  in 
recent  years  relative  to  other  training  accounts.  Reduced  funding  potentially  means  that 
implementation  of  ALM  will  focus  developers  on  reusing  existing  IMI  and  on  scaling  back 
development  of  new  IMI.  As  a  consequence,  developers  are  concerned  that  they  will  need  to 
develop  lower  bandwidth  courses,  sacrificing  the  latest  multimedia  features  (e.g.,  videos, 
complex  animation,  or  high  resolution  graphics).  These  features  are  often  viewed  as  enhancing 
students’  experiences,  and  consequently,  students’  interest  in  what  they  are  learning. 

However,  Straus  et  al.  (2011)  suggested  an  alternative  perspective.  What  they  found  was 
that  students  were  not  necessarily  more  engaged  by  data-intensive,  high  bandwidth  media;  they 
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were  engaged  by  training  that  exhibited  well-thought-out  learning  principles,  i.e.,  good  pedagogy 
and  good  content.  The  training  first  has  to  be  carefully  designed  for  learning,  and  second,  for 
visual  appeal.  A  focus  on  the  structure  and  flow  of  information,  careful  use  of  part-task  and 
whole-task  examples,  etc.,  when  developing  future  IMI  may  help  to  improve  student 
engagement,  even  in  an  era  when  there  are  fewer  resources  available  to  dedicate  to  IMI 
development. 

The  ALM  points  to  an  expanded  role  for  IMI  in  Anny  training  and  education.  To  address 
this  expanded  role,  as  noted  by  Straus  et  al.  (2011),  the  TRADOC  Capabilities  Manager  for  dL 
(TCM-DL)  has  outlined  plans  to  change  how  IMI  is  developed  and  delivered,  moving  from  a 
client-server  paradigm — with  training  delivered  on  desktop  computers — to  cloud  computing  (i.e., 
services  provided  via  the  internet),  using  mobile  learning  (mLearning)  devices.  The  TCM-DL 
has  also  envisioned  a  shift  from  “long  courses  with  protracted  development  time  frames  to 
‘chunked  content’  that  can  be  developed  rapidly”  (p.  1).  This  centralized  model  of  IMI 
development,  storage,  maintenance,  and  distribution  could  significantly  change  how  the  Army 
develops  and  uses  IMI. 

Even  so,  these  issues  do  tend  more  toward  technical  considerations  than  issues  of 
instructional  design  and  principles  of  learning.  Optimally,  what  the  Anny  may  need  is  an 
organizational/delivery  structure  that  directly  incorporates  labels  to  indicate  point  of  need 
content.  As  IMI  is  developed  by  topic  and  point  of  need  type,  it  could  be  incorporated  into  the 
organizational/delivery  fonnat,  e.g.,  familiarization  or  core  training  on  “Call  for  Fire/Adjust  Fire 
for  Squad  Feaders,”  or  tailored  training  on  “Squad  Urban  Defense.”  The  different  point  of  need 
variation  could  be  developed  simultaneously  as  the  proponent  sees  a  need  for  a  particular  topic, 
reusing  some  images,  text,  etc.,  between  different  point  of  need  variations  (Jean  Dyer,  personal 
communication,  29  May  2013). 

What  Can  Be  Reused  in  Moving  to  a  Point  of  Need  IMI  Design? 

When  conducting  this  research,  we  established  a  standard  hierarchy  for  defining  the 
structure  of  IMI  courseware  (i.e.,  course,  lesson,  module,  and  SCO).  Using  this  structure,  we 
found  that  the  greatest  efficiencies  are  gained  through  the  reuse  or  repurposing  of  modules  or 
smaller  “chunks”  of  information.  Complete  lessons  were  generally  broad  in  scope  and  did  not 
address  any  specific  needs.  Typically  the  lesson  covered  more  than  was  necessary  for 
familiarization  and  lacked  the  details  required  for  core-type  training.  Moreover,  lessons 
generally  did  not  have  a  structure  that  allowed  students  to  determine  how  they  completed  the 
lesson  nor  to  make  informed  decisions  about  what  parts  of  the  lessons  they  might  complete.  For 
IMI  to  address  point  of  need  it  should  be  structured  in  such  a  way  that  the  learner  can  easily 
move  through  the  material  to  selectively  address  portions  of  a  course  that  focus  on  his  or  her 
present  learning  needs.  Current  SCOs  do  not  seem  to  be  designed  to  facilitate  this  type  of 
movement  through  the  content. 


s  While  not  explained  in  the  RAND  Study  (Straus  et  al.,  2011),  “chunked  content’’  refers  to  stand-alone  learning 
modules  that  can  be  completed  in  a  short  duration.  Each  module  would  contain  the  necessary  instructional  and 
assessment  materials.  Given  that  each  "chunk”  is  relatively  short,  these  modules  could  be  developed  more  rapidly 
than  lengthy  courses  of  a  much  longer  duration  with  expanded  content. 
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With  respect  to  point  of  need,  reuse  or  repurposing  of  individual  SCOs  seemed  to  provide 
the  least  efficiencies.  The  best  way  to  explain  this  inefficiency  is  via  an  example  (see  Figure  2). 
Consider  SCOs  to  be  all  of  the  components  that  go  into  an  automobile  (e.g.  motor,  transmission, 
tires,  seats)  as  well  as  the  thousands  of  minor  parts  (e.g.,  bolts,  wires,  cables,  gaskets,  seals)  that 
are  necessary  to  connect  the  components  and  make  the  entire  automobile  operate.  These 
components  and  parts  fit  perfectly  in  the  existing  automobile,  because  they  were  chosen, 
designed,  developed,  and  assembled  to  fit  the  specific  automobile.  However,  attempting  to 
remove  any  one  or  some  of  these  components  for  reuse  in  a  different  automobile  could  be  quite 
time  consuming  and  rather  ineffective.  For  example,  the  transmission  might  be  of  a  great 
quality,  but  is  it  the  right  dimension  to  fit  within  a  different  frame,  does  it  have  the  required  gear 
ratio,  and  do  the  mounting  bolts  synchronize  (e.g.,  location,  size,  thread  count)  with  the  new 
engine?  While  the  existing  automobile  and  its  components  work  well  for  its  intended  purpose, 
just  like  existing  IMI  also  works  well,  when  attempts  are  made  to  take  it  apart  for  reuse,  the 
components  most  likely  will  not  fit  properly.  This  is  especially  the  case  here,  where  we  sought 
to  use  IMI  designed  to  reach  a  broad  audience  and  repurpose  it  to  address  the  needs  of  a  specific 
audience  or  even  an  individual  learner. 


Determining  the  Suitability 

of  Existing  Course  Content 


Existing  IMI 


Figure  2.  Can  Existing  IMI  Content  Be  Reused  to  Meet  Point  of  Need? 

Even  if  the  components  and  parts  fit,  they  might  not  function  as  intended.  Just  as  the 
computerized  devices  and  gauges  monitor  and  control  automobile  functioning,  in  most  existing 
IMI  there  is  a  learning  management  system  (LMS)  that  is  hosting  the  IMI.  This  LMS  tracks 
student  access,  progress,  and  performance.  It  also  serves  as  the  main  controller  to  have  the  IMI 
function  properly.  When  you  take  IMI  components  from  an  existing  course,  which  has  LMS- 
integrated  hooks  and  tracking  mechanisms  embedded,  you  must  be  able  to  identify  these  so  they 
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can  be  removed  or  changed  to  function  properly  in  the  reused  version  of  the  IMI.  Even  though 
an  existing  IMI  component  might  be  high-quality,  and  the  LMS  controls  can  be  changed,  a  key 
question  still  remains.  Does  the  existing  IMI  satisfy  the  point  of  need  the  new  IMI  is  attempting 
to  address? 

The  tremendous  challenge  in  trying  to  reuse  existing  IMI  involves  multiple  issues.  First 
is  the  exhaustive  search  required  to  locate  the  potential  SCOs  to  meet  the  point  of  need  of  the 
new  IMI.  Then  comes  the  complex  challenge  of  being  able  to  modify  the  SCO  to  function 
properly  within  the  new  IMI.  Finally,  the  resultant  new  IMI  must  address  the  desired  point  of 
need,  which  is  not  what  the  original  IMI  was  designed  to  do. 

Physical  review  of  IMI  content  is  currently  the  only  method  for  selection  of  reusable 
content.  Even  though  the  Sharable  Content  Object  Reference  Model  (SCORM)  has  established 
policies  for  tagging  and  identifying  reusable  objects,  there  is  no  centralized  repository  and  no 
means  to  search  existing  IMI  courseware  to  identify  potential  SCOs  to  meet  very  specific  points 
of  need.  The  current  method  is  to  conduct  a  visual  and/or  audio  check  of  SCOs  and  pieces.  Not 
only  can  this  be  extremely  time-consuming  and  inefficient,  but  obtaining  access  to  IMI  source 
files  where  SCOs  can  be  examined  might  not  be  possible.  While  libraries  of  functioning  IMI 
courseware  are  accessible,  the  IMI  source  files  (e.g.,  SCOs,  such  as  narrations,  graphic  images, 
and  animations)  are  often  not  available.  Many  times  this  material  must  be  obtained  from  the 
source,  the  individual  or  group  who  developed  the  IMI  prior  to  the  software  being  compiled. 

The  preponderance  of  existing  IMI  courseware  was  designed  and  developed  to  operate 
within  some  LMS.  This  requires  the  integration  of  coding  and  functionality  that  is  often 
transparent  to  the  user  or  viewer  of  the  IMI  courseware.  However,  these  codes  are  built-in  to  the 
pages  and  SCOs  of  the  IMI.  When  extracting  components  from  existing  IMI  courseware,  care 
must  be  exercised  to  check  for  these  codes  and  functions.  These  hidden  items  can  cause  the 
components  to  operate  differently  than  anticipated  and  would  likely  be  inconsistent  if  run  in  a 
different  LMS.  Integrating  lessons,  modules,  chunks,  or  SCOs  into  an  LMS  that  differs  from  the 
original  normally  requires  some  technical  modification  or  tweaking. 

Source  file  availability.  Source  file  availability  includes  determination  if  the  source  files 
are  available  and  if  the  text  and  media  files  are  searchable  and  extractable.  Source  files  may 
include  access  to  storyboards  and/or  the  instructional  media  design  package  (IMDP)  (TRADOC, 
2003)  which  captures  the  design,  conventions  and  standards,  and  instructional  strategies  required 
by  the  school,  proponent,  or  supported  instructional  agency.  The  IMDP  may  also  include  a 
description  of  the  proposed  training  program,  lesson  development  support  information,  and  wire¬ 
frame  diagrams  that  identify  Tenninal  Learning  Objectives  (TLOs),  Enabling  Learning 
Objectives  (ELOs),  and  SCOs  that  could  support  reuse  decisions.  The  IMDP  provides  insights 
into  the  objectives  of  the  designers  and  authors  and  may  disclose  how  they  intended  to  employ 
the  media,  materials,  exercises,  and  tests  to  guide  learning,  convey  knowledge,  and/or  attain 
desired  levels  of  proficiency  or  performance.  As  a  minimum,  source  files  should  provide 
original  graphics,  media,  and  text  files.  These  source  files  should  be  relatable  and  searchable, 
supporting  efforts  to  locate  and  identify  files  and  elements  of  the  existing  IMI  that  may  have 
potential  for  reuse.  Files  drawn  from  executable  files  on  CD-ROM  course  materials  or 
downloaded  from  a  server  across  the  internet  or  a  network  may  not  have  sufficient  quality  or 
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resolution  for  reuse.  These  files  may  also  be  integrated  or  woven  into  the  LMS  of  the  existing 
courseware.  Such  files  may  not  be  reusable  without  further  manipulation. 

Doctrinal  accuracy  and  currency.  Doctrinal  correctness  and  currency  are  crucial 
factors.  While  some  military  doctrinal  principles  rarely  change,  doctrinal  tenns  and  tactics, 
techniques,  and  procedures  (TTP)  frequently  do.  The  focus  of  the  operating  environment,  heavy 
in  counterinsurgency  and  urban  operations,  the  introduction  of  new  technologies,  changes  in 
Army  leadership,  and  greater  emphasis  on  joint  and  combined  operations  introduce  a  number  of 
new  or  modified  terms.  A  number  of  TTP  can  therefore  be  modified  or  introduced.  These  TTP 
conform  to  commonly  prescribed  rules  of  engagement,  incorporate  new  technologies,  emphasize 
precision  room  clearing  techniques,  and  the  increasing  lethality  of  improvised  explosive  devices 
(IEDs). 


With  each  change  in  TTP  or  equipment,  new  training  has  to  be  introduced.  Some  aspects 
of  training  will  remain  unchanged  between  different  variations  of  equipment,  with  only  minor 
variations  in  procedures,  etc.,  becoming  relevant  with  the  new  equipment.  Often,  Soldiers  will 
judge  the  currency  and  relevance  of  a  piece  of  IMI  or  other  learning  resource  based  on  factors 
that  may  not  directly  reflect  the  currency  of  the  training  content.  For  instance,  training  may  be 
viewed  as  irrelevant  if  it  is  being  distributed  on  VHS  tapes  rather  than  online  or  on  DVDs. 
Moreover,  it  is  important  to  consider  how  Soldiers  may  react  to  images  presented  within  the  IMI. 
For  instance,  outdated  uniforms  are  a  common  complaint  which  leads  Soldiers  to  view  the  IMI  as 
irrelevant,  even  if  the  IMI  content  is  in  fact  doctrinally  accurate  and  correct.  For  Soldier 
acceptance,  graphics  and  visual  depictions  must  show  current  uniforms  and  equipment.9  The 
appearance  of  uniforms  in  IMI  other  than  the  current  ones  adversely  dates  materials  and 
undermines  the  courseware  credibility.  Except  when  used  for  contrast  or  historical  examples, 
doctrine,  weapons,  equipment,  and  TTP  depicted  in  IMI  need  to  be  current.  It  is  essential  to  vet 
materials  with  current  doctrinal  references  and  subject  matter  experts.  Graphics,  examples,  and 
illustrations  should  depict  Soldiers  in  current  and  correctly  worn  uniforms  with  up-to-date 
equipment. 

Suitability  for  reuse/repurposing.  Designers  encounter  a  variety  of  problems  when 
developing  IMI.  The  interaction  one  usually  has  with  students  in  a  classroom  is  not  present. 
Therefore,  IMI  designers  must  gauge  potential  difficulties  students  may  have  with  the 
presentation  content  or  style.  In  fact,  for  some  who  are  accustomed  to  writing  in  a  very 
academic  style,  transitioning  to  the  more  infonnal  tone  of  IMI  can  be  challenging.  While  these 
areas  can  be  overcome  when  designing  and  developing  IMI  from  scratch,  this  becomes  a  key 
issue  when  considering  IMI  for  reuse/repurposing. 

Swales’  (2000)  research  provides  additional  recommendations  and  guidance  to  teachers, 
instructional/course  designers,  and  other  education  professionals  (elementary  to  university). 


9  For  example,  the  Universal  Camouflage  Patterned,  Army  Combat  Uniform  (ACU)  replaced  the  forest  (dark  green, 
black,  brown,  and  dark  tan)  patterned  Battle  Dress  Uniform  (BDU)  and  desert  (light  tan,  pale  green,  and  brown)  and 
earlier  chocolate-chip  (light  tan,  pale  olive  green,  brown,  with  clusters  of  black-on-white  spots)  patterned  Desert 
Camouflage  Uniform  (DCU)  worn  in  the  1980s  and  1990s.  Additionally,  modified  ACU  MultiCam,  7-color 
camouflage  pattern,  designed  to  conceal  the  wearer  in  varied  environments,  seasons,  elevations,  and  light  conditions 
has  been  issued  since  2010  to  Special  Operations  Forces  and  units  deploying  to  Afghanistan. 
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Swales  presented  stakeholder  groups  with  “distance  education  materials”  (both  paper  based  and 
e-learning)  for  review.  The  stakeholder  groups  advised  that  new  or  purpose-built  materials  as 
well  as  reused  or  repurposed  materials  should  be  focused  on  particular  audiences.  The  guidance 
also  encouraged  the  inclusion  of  learning  activities  to  assist  in  breaking  the  content  into  suitable 
learning  blocks,  a  similar  finding  to  that  of  Dyer  and  Salter  (2001).  Further,  the  materials  should 
encourage  and  motivate  the  learner,  as  well  as  provide  feedback  to  enable  the  learner  to  gauge 
understanding  and  progress. 

While  content  and  style  varied  between  the  courses,  much  of  the  existing  IMI  we 
examined  did  not  seem  easily  reusable  in  a  point  of  need  design  context.  For  example,  most  of 
the  existing  IMI  we  evaluated  did  not  provide  examples,  did  not  integrate  regular  testing  to  check 
knowledge  at  the  end  of  topics,  and  provided  courses  or  lessons  that  were  general  in  nature.  The 
IMI  tended  to  follow  a  “one-size-fits-all”  design  strategy,  rather  than  focusing  on  specific  topics 
with  an  appropriate  depth  of  information  tailored  to  specific  points  of  need.  While  some  parts  of 
existing  IMI  can  be  reused  or  modified,  a  redesign  effort  will  likely  require  significant 
development  of  new  content  to  address  different  variations  of  point  of  need. 

Conclusion 

As  the  Army  continues  to  implement  ALM,  a  distinct  shift  in  the  Anny’s  approach  to 
education  and  training  is  becoming  apparent.  One  area  in  which  this  shift  is  exemplified  is  in  the 
design  and  development  of  IMI.  Much  of  the  existing  IMI  was  designed  to  be  applicable  to  the 
broadest  possible  audience.  To  develop  IMI  that  is  oriented  to  a  learner-centric  approach 
requires  the  application  of  point  of  need  concepts  and  tailored  training  techniques.  These 
techniques  will  enhance  the  ability  to  address  individual  differences  in  learning  needs  in  IMI. 

At  the  outset  of  this  research,  we  were  focused  on  how  to  apply  point  of  need  concepts 
and  tailored  training  techniques  in  existing  Army  IMI.  It  became  apparent  that  to  accomplish  a 
large-scale  revision  of  Army  IMI  to  address  point  of  need  would  likely  be  prohibitive  in  terms  of 
time  and  costs.  The  most  reasonable  course  of  action  may  be  to  focus  on  applying  point  of  need 
and  tailored  training  techniques  in  the  future  IMI,  at  the  outset  incorporating  key  principles  and 
techniques  into  its  design. 

The  approaches  we  described  in  this  research  would  present  good  value  to  the  Army,  and 
more  importantly,  to  Soldiers  who  are  using  IMI  resources  to  acquire  and  maintain  their  knowledge 
and  skills.  The  techniques  we  identified  and  discussed  are  not  expensive  and  bandwidth-heavy.  The 
Army  should  continue  to  focus  its  efforts  on  the  hallmarks  of  carefully-executed  design  of  IMI  to 
enhance  learning  and  to  allow  Soldiers  to  engage  with  IMI  in  the  sequence  and  depth  that  they  desire. 
Overuse  of  bandwidth-heavy  multimedia  may  in  fact  distract  from  the  key  knowledge  and  skills 
being  taught. 

To  implement  the  ALM  learner-centric  concept,  the  Army  needs  to  develop  IMI  that  is 
engaging  and  designed  to  give  learners  both  structure  and  autonomy  in  their  learning  experiences.  It 
should  also  focus  learners  on  evaluating  their  progress  in  order  to  make  informed  decisions  about 
their  own  learning  needs,  and  to  reflect  on  their  learning  process.  The  principles  and  techniques  we 
identified  in  this  research  support  the  implementation  of  the  Army’s  learner-centric  model  to  address 
individual  Soldiers  in  learning  at  their  point  of  need. 
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Acronyms 


ACC 

ACU 

ADA 

ADL 

AFQT 

AKO 

ALC 

ALM 

ATN 

Army  Capstone  Concept 

Army  Combat  Uniform 

Air  Defense  Artillery 

Advanced  Distributed  Learning 

Anned  Forces  Qualification  Test 

Army  Knowledge  Online 

Army  Learning  Concept 

Army  Learning  Model 

Army  Training  (and  Education)  Network 

BDU 

Battle  Dress  Unifonn 

CoE 

CORDA 

Center  of  Excellence 

Content  Object  Repository  Registration/Resolution  Architecture 

DCU 

dL 

DLDC 

DoD 

DOTD 

Desert  Camouflage  Unifonn 

Distributed  Learning 

Distributed  Learning  Development  Center 

Department  of  Defense 

Directorate  of  Training  and  Doctrine 

ELO 

Enabling  Learning  Objective 

FA 

FCoE 

Field  Artillery 

Fires  Center  of  Excellence 

IED 

IMDP 

IMI 

Improvised  Explosive  Device 

Instructional  Media  Design  Package 

Interactive  Multimedia  Instruction 

LCMS 

LMS 

Learning  Content  Management  System 

Learning  Management  System 

MCoE 

MFE 

MIMIC 

MOS 

MT2 

Maneuver  Center  of  Excellence 

Maneuver,  Fires,  and  Effects 

Military  Interactive  Multimedia  Instruction  Center 

Military  Occupational  Specialty 

MyTraining  Tab 

SCO 

SCORM 

SUGV 

Sharable  Content  Object 

Sharable  Content  Object  Reference  Model 

Small  Unmanned  Ground  Vehicle 
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TADLP 

TCM-DL 

TLO 

TRADOC 

HP 

VEILS 


The  Army  Distance  Learning  Program 
TRADOC  Capabilities  Manager  for  dL 
Terminal  Learning  Objective 
U.S.  Army  Training  and  Doctrine  Command 
Tactics,  Techniques,  and  Procedures 

Virtual  Experience  Immersive  Learning  Simulation 
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Appendix  A 


Maneuver,  Fires,  and  Effects  Courseware  Reviewed 

The  Maneuver  Center  of  Excellence  (MCoE)  and  the  Fires  Center  of  Excellence  (FCoE), 
provided  current  IMI  materials  for  our  examination.  These  materials  included: 

•  MCoE  IMI  Courseware 

o  United  States  Army  Infantry  School  (USAIS)  Improvised  Explosive  Device  (IED) 
Defeat  Course.  Lessons  include: 

■  Course  Introduction. 

■  React  to  a  Possible  Improvised  Explosive. 

■  Plan  for  an  Improvised  Explosive  Device  Threat. 

■  React  to  an  Improvised  Explosive  Device  Attack. 

■  Conduct  Tactical  Site  Exploitation  in  a  Counterinsurgency  Environment. 

■  Conduct  Battle  Staff  Operations  Process  in  a  Counterinsurgency 
Environment. 

o  Total  Army  Training  System  19K  Advanced  Leaders  (TATS  19K  ADV  LDR) 
Course,  Phase  I.  Lessons  include: 

■  Course  Introduction 

■  Army  Aviation 

■  Conduct  Operations  with  Attached  Infantry 

■  Direct  Convoy  Escort  Operations 

■  Patrolling  Operations  with  Urban  Considerations 

■  Cordon  and  Search  (kilo) 

■  Conduct  a  Raid  (kilo) 

■  Army  Writing 

■  Army  Green 

■  Cultural  Awareness 

■  Counterinsurgency  Operations 

■  Branch  History  and  Heritage 

■  Combat  Identification 

■  Contemporary  Operating  Environment 

■  Supervise  PMCS  and  Maintenance  Forms  -  Kilo 

■  Recognition  of  Combat  Vehicles 

■  Conduct  an  After- Action  Review 

■  Warrior  Transition  Unit 

■  Training  Management 

■  Platoon  Combatives 

o  19K  Military  Occupation  Specialty  -  Transition  (reclassification  training)  (19K 
MOS-T)  Course,  Phase  I  (dL).  Lessons  include: 

■  Identify  Topographic  Symbols  on  a  Military  Map 

■  Determine  the  Grid  Coordinates  of  a  Point  on  a  Military  Map 

■  Determine  the  Elevation  of  a  Point  on  the  Ground 

■  Identify  Terrain  Features  on  a  Military  Map 

■  Detennine  a  Magnetic  Azimuth  Using  a  Compass 

■  Orient  a  Map  Using  a  Compass 
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■  Navigate  While  Dismounted 

■  Navigate  While  Mounted 

■  Enter  a  Building  During  an  Urban  Operation 

■  Clear  a  Building 

■  Perform  Movement  Techniques 

■  Prepare  Individual  and  Crew-Served  Positions 

■  Use  Visual  Signaling  Techniques 

■  Recognize  Armored  Vehicles  and  Aircraft 

■  Estimate  Range 

■  Operate  the  Automated  Net  Control  Device 

■  Operate  the  Precision  Lightweight  GPS  Receiver 

■  Operate  the  Defense  Advanced  GPS  Receiver 

■  Maintain  Operator’s  Part  of  Equipment  Record  Folder 

■  Ml A1  Familiarization  (Hull) 

■  M1A1  Familiarization  (Turret) 

■  Use  the  M1A1  Operator’s  Manuals 

■  Conduct  M1A1  Hull  PMCS 

■  Conduct  Ml A1  Turret  PMCS 

■  Use  Ml A1  Operator’s  Controls 

■  Troubleshoot  Using  the  Driver’s  Control  Panel 

■  Prepare  the  Gunner’s  Station  for  Operation 

■  Perform  Loader’s  Before  Operation  Checks 

■  Perform  Loader’s  After  Operation  Checks 

■  Inspect  120mm  Ammunition  for  Serviceability 

o  TATS  19D  ADV  LDR  Course,  Phase  I.  Lessons  include: 

■  Course  Introduction 

■  Conduct  a  Linkup/Relief  in  Place 

■  Army  Aviation 

■  Conduct  an  Area/Zone  Reconnaissance 

■  Direct  Convoy  Escort  Operations 

■  Patrolling  Operations  with  Urban  Considerations 

■  Cordon  and  Search  (delta) 

■  Conduct  a  Raid  (delta) 

■  Army  Writing 

■  Army  Green 

■  Cultural  Awareness 

■  Counterinsurgency  Operations 

■  Branch  History  and  Heritage 

■  Combat  Identification 

■  Contemporary  Operating  Environment 

■  Supervise  PMCS  and  Maintenance  Forms  -  Delta 

■  Recognition  of  Combat  Vehicles 

■  Conduct  an  After-Action  Review 

■  Warrior  Transition  Unit 

■  Training  Management 

■  Platoon  Combatives 
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o  19D  MOS-T  Course,  Phase  I.  Lessons  include: 

■  Identify  Topographic  Symbols  on  a  Military  Map 

■  Detennine  the  Grid  Coordinates  of  a  Point  on  a  Military  Map 

■  Determine  the  Elevation  of  a  Point  on  the  Ground 

■  Identify  Terrain  Features  on  a  Military  Map 

■  Measure  Distance  on  a  Military  Map 

■  Determine  a  Magnetic  Azimuth  Using  a  Compass 

■  Orient  a  Map  Using  a  Compass 

■  Detennine  Azimuths  Using  a  Protractor 

■  Compute  Back  Azimuths 

■  Navigate  While  Dismounted 

■  Navigate  While  Mounted 

■  Enter  a  Building  During  an  Urban  Operation 

■  Clear  a  Building 

■  Perfonn  Movement  Techniques 

■  Prepare  Individual  and  Crew-Served  Positions 

■  Select  Hasty  Firing  Positions 

■  Direct  Vehicle  Tactical  Movement 

■  Conduct  an  Area  Reconnaissance  by  a  Platoon 

■  Conduct  an  Area/Zone  Reconnaissance 

■  Perfonn  a  Reconnaissance  by  Fire 

■  Conduct  Actions  at  a  Danger  Area 

■  Use  Visual  Signaling  Techniques 

■  Recognize  Armored  Vehicles  and  Aircraft 

■  Estimate  Range 

■  Perfonn  Surveillance  Without  Electronic  Devices 

■  Collect  Route  Classification  Data 

■  Call  For  and  Adjust  Indirect  Fire 

■  Identify  Threat  Weapons 

■  Operate  Night  Vision  Goggles  AN/PVS-7 

■  Maintain  Night  Vision  Goggles  AN/PVS-7 

■  Operate  the  Automated  Net  Control  Device 

■  Operate  the  Precision  Lightweight  GPS  Receiver 

■  Operate  the  Defense  Advanced  GPS  Receiver 

■  Maintain  Operator’s  Part  of  Equipment  Record  Folder 

o  Small  Unmanned  Ground  Vehicle  (SUGV)  Leaders  Course  (Training  Prototype) 
o  SUGV  Employment  for  Leaders  -  Platoon 
o  Welcome  to  Jump  Master 
o  Welcome  to  Path  Finder 

o  A  Day  in  the  Bam  (a  Virtual  Experience  Immersive  Learning  Simulation  [VEILS 
®])  ' 

■  Annor  Platoon  Sergeant  Module 

■  Infantry  Platoon  Sergeant  Module 
•  FCoE  IMI  Courseware  -  (13F  MOS  Lessons) 

o  Construct  a  Laser  Range  Safety  Fan 
o  Conduct  a  Suppression  Mission 
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o  Conduct  an  Immediate  Suppression  Mission 
o  Conduct  Fire  for  Effect  Mission 
o  Conduct  Coordinated  Illumination 
o  Conduct  Final  Protective  Fires 
o  Conduct  Immediate  Smoke  Mission 
o  Establish  an  Observation  Post 
o  Conduct  a  Mortar  Registration 
o  Request  Fire  on  Irregularly  Shaped  Targets 
o  Process  Platoon  Forward  Observer  Target  List 
o  Plan  Occupation  of  an  Observation  Post 
o  Process  Observer  Target  List 
o  Clear  Indirect  Fires 

o  Engage  Targets  with  Close  Air  Support  (CAS) 
o  Coordinate  Passage  of  Lines 
o  Engage  a  Moving  Target  with  Indirect  Fire 
o  Prepare  a  Company  Fire  Support  Plan 
o  Conduct  a  Highburst  Registration 
o  Engage  Targets  with  Naval  Surface  Support 

Included  in  the  above  FCoE  lessons  modules/lessons  are  background  prerequisite  lessons  and 
lessons  on  foundation  tasks  and  skills  that  may  be  required  to  understand  and  master  the  primary 
lessons.  These  include: 

o  Fire  Planning 
o  Threat  Recognition 
o  Determine  Target  Location 
o  Call  for  Fire  (Basic) 
o  Subsequent  Corrections 
o  Prepare  a  Terrain  Sketch 
o  Fire  Support  Coordination  Measures  (FSCMs) 
o  Laser  Safety 

Subjects  available  through  the  Army  Knowledge  Online  (AKO)  newly  established  MT2  site 
included: 

o  Engage  Targets  During  an  Urban  Operation 
o  Prepare  a  Range  Card  for  a  Machine  Gun 
o  Perform  a  Function  Check  on  an  MK19  Machine  Gun 
o  Unload  an  MK19  Machine  Gun 

o  Mount  a  Night  Vision  Sight,  AN/TVS-5,  on  an  MK19  Machine  Gun 
o  Prepare  Positions  for  Individual  and  Crew-Served  Weapons  During  an  Urban 
Operation 

o  Enter  a  Building  During  an  Urban  Operation 
o  Correct  Malfunctions  on  an  MK19  Machine  Gun 
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Appendix  B 


Initial  Criteria  for  Evaluating  Existing  IMI 

Criterion  Description 

Complexity/Depth  of  Information  Is  there  sufficient  content  to  provide  the  desired  coverage  of  the  subject  area, 

Presented  especially  when  attempting  to  form  the  different  points  of  need  training 

material? 

Formal/Doctrinal  Correctness  Is  there  anything  that  would  raise  subject  matter  experts’  and/or  trainers’ 

(Experts’  Perspective)  concerns  with  the  currency  and  accuracy  of  the  material?  Are  there  potential 

negative  transfer  issues? 

Repurposable  Graphics/Images  Are  there  alternative  principles/ideas  demonstrated  by  the  same 

image/graphic?  Are  the  graphics/images  contextualized  to  guide  how  a 
Soldier  is  interpreting  them? 

Verisimilitude/Face  Validity  Would  Soldiers  see  the  information  and  images  presented  as  being  accurate 

(Learners’  Perspective)  representations  of  current  Army  knowledge  and  practice  (e.g.,  current 

doctrine,  up-to-date  uniforms,  and  current  weapon  systems)? 

Viable  Examples  How  are  examples  being  used  and  are  there  single  or  multiple  examples? 

Multiple  examples  may  be  better  to  allow  Soldiers  to  generalize  principles 
by  deriving  consistent  patterns  across  examples  (cf.  Schwartz  &  Bransford, 
1998). 

Narrative  Flow  Does  the  narrative  of  the  training  make  sense?  Do  instructional  pages  that 

come  later  logically  build  on  what  came  before? 

Presentation  is  Focused  vs.  Do  the  parts  have  a  clear  topical  focus,  or  does  the  training  meander? 

Diffuse 

Outcome  Meets  Goal  Does  the  training  reasonably  appear  to  accomplish  established  goals? 

Grouping  of  Content  Are  the  modules  and  information  grouped  in  a  way  that  makes  sense  and 

provides  a  coherent  structure  (i.e.,  support  development  of  schemas)? 

Appropriate  Testing  Do  the  tests  legitimately  cover  the  material  at  a  conceptual  level,  or  are  they 

focused  on  insignificant  details  and  off  the  subject  area? 

Interactivity/Control  In  what  ways  is  the  Soldier  being  asked  to  interact  with  the  training  package, 

materials,  etc.?  Is  the  interactivity  distracting  to  or  supportive  of  the  overall 
goals  of  the  training?  Is  the  Soldier  given  a  sense  of  being  able  to  shape 
his/her  own  learning  process? 

Timing  Would  a  Soldier  be  able  to  develop  understanding  of  one  learning  point 

before  the  next  one  is  presented?  Are  there  logical  points  at  which  Soldiers 
can  take  a  break  from  what  they  are  learning? 

Use  of  Prior  Knowledge  Is  prior  knowledge  elicited  or  refreshed  before  it  is  built  upon? 
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Technical  Characteristics 


Suitability  for  Tailoring 


Are  the  requisite  files  available  and  in  a  format  to  be  able  to  pull  apart  the 
training  package?  Is  the  software  package  that  was  used  in  developing  the 
IMI  courseware  current  and  useable?  Are  the  respective  pieces  of  the 
courseware  files  able  to  be  reconfigured  within  more  current  software? 

Does  the  courseware  include  aspects  that  would  support  various  tailoring 
techniques  (e.g.,  prompts  based  on  thematic  model,  structuring  presentation 
of  elaborated/basic  vs.  advanced  material,  color  cuing,  pretesting  and 
modifying  learning  presentation  based  on  performance) 
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